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ABSTRACT 
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PREFACE 



Under contract to the Office of Human Development Services, Administration on Developmental 
Disabilities (ADD), Bonnie Walker and Associates (BWA) has conducted a study to develop a fire 
safety certification system which can deliver uniform, validated training to board and care 
operators who work with adults with developmental disabilities. This study expands on previous 
work performed by BWA documented in "A Fire Safety Certification System for Board and Care 
Operators, Phase I, Pinal Report, December 20, 1988.* 

Meeting the objectives of the study required the development of a workshop to train instructors to 
conduct the fire safety workshop for board and care providers developed during Phase I and 
conducting a large scale field test of the provider's workshop using these newly trained 
instructors. Data were collected through a number of measures: pre- and post tests, course 
evaluation forms, follow-up surveys with instructors who had completed training, videotape 
recordings of the Instructor's Workshop and of a Focus Group meeting during which Advisory 
Council members discussed their observations of the Instructor's Workshop and the fieldtest of the 
Provider's Workshops. 

Bonnie L. Walker was the principal investigator. Susan Shemanski performed a mcgor portion . 
of the data analysis. Other staff contributors were Margaret S. Withrow, April L. Walker, and 
Alexis Smith. Advisory Council members who contributed by conducting portions of the 
Instructor's Workshop, conducting workshops for providers, or observing provider workshops, or 
assisting in the data collection and reporting were: Terezie Bohrer, Harry S. Bradley, John 
Bryan, Harold D. Hicks, Jr., Marjory Owens, Betty Jo Mayeske, Michael Strait. 

We wish to recognize the role played by Marjory Owens, Housing Specialist for Project Home, and 
the Project Home case workers who participated in the Instructor's Workshop and conducted the 
eight workshops during the field test. Ms. Owens organized all of the field test workshops, 
recruited providers to participate in them, conducted a major part of the fire safety instruction, and 
provided technical assistance throughout the development of the Instructor's and Provider's 
Workshops. 

We also wish to recognize the role played by the study's technical monitor, Vernard Evans of the 
Administration on Developmental Disabilities. She followed our work closely and could be 
counted on throughout to provide meaningful suggestions and assistance when they were needed. 
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INTRODUCTION AND EXECUTIVE SUMMARY 



Under contract to the Administration on Developmental Disabilities, Bonnie Walker & 
Associates has completed the pilot test of the Instructor's Workshop and the field test of the 
Provider's Workshop,' two components of the Fire Safety Certification System for Board and Care 
Operators. This study was intended to produce a uniform, validated system for delivering fire 
safety training to board and care providers nationally who serve adults with developmental 
disabilities. 

Phase I Results 

Phase I took place between June and December 1988. During that period, BWA developed the 
curriculum and materials for a fire safety workshop for board and care operators. Materials 
included a trainer's manual, participant's coursebook, two videotapes, an audiocassette, eleven 
job aids, a pre- and post test, and other evaluation instruments. 

On November 14, 1988, the materials were pilot tested at Melwood Training Center, a non-profit 
organization that operates residential care facilities for adults with developmental disabilities in 
Maryland* Participants were direct care and supervisory staff members of that organization. 
Results of the evaluation indicated a statistically significant mean gain from pre- to post test for 
the participants (N=12). Participants improved as a group on a large majority of the test items 
(N=21). Results of the course evaluation which measured the participants' feelings about the 
training indicated that participants had a very positive view of their own learning. Participants 
were surveyed to determine their interest in continuing education materials. A large majority 
indicated an interest in learning more about fire safety. Additional data were collected by an 
independent observer who recorded training events at five minute intervals, an audiocassette 
recording of the training, and by a second observer who made unstructured comments about 
problems and successes throughout the training and noted suggested changes to the Trainer's 
Manual and Coursebook. 

Information that would be useful in revising the materials was sought from the pre- and post test 
results, the course evaluation results, from observations during the training, and from a review of 
the pilot test materials by members of the Focus Group. The findings suggested that combining the 
Trainer's Manual and the Coursebook into one document for the trainers would promote ease in 
handling. Other suggestions included improving some of the photographs in the coursebook, 
producing additional videos, and converting the open-ended test format to a multiple choice test for 
the field test planned for Phase II. 

Phase II Objectives 

The objectives of the current research where to: . 

c Develop and pilot test a workshop to train instructors to present the provider's workshop, 
and 

• Conduct a field test of the provider's workshop. 

Data Collection and Results 

Pilot Test of the Instructor's Workshop 

A two-day workshop for training instructors was developed and pilot tested on June 5 and 6, 1989. 
Nineteen people participated. Of those, sixteen complete sets of data were obtained. Participants 
were instructed in the contents of the provider's workshop materials and given additional 
background information in fire safety, developmental disabilities, board and care operations 
and training techniques. Pre- and post tests were administered. Results showed an average 
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group gain of 38% percentage points. Course evaluations completed by the group suggested a strong 
overall approval of the workshop presentation, content, and training aids. Following the 
workshop, the twelve of the sixteen eligible participants successfully conducted workshops for 
board and care providers at eight sites in Maryland and completed their requirements for 
certification. A survey of those instructors who had completed a workshop indicated that they 
approved overall of their training, the Instructor's Manual, the course materials and believed that 
the providers both enjoyed the workshop and learned from it. 

Field Test of the Fire Safety Workshop for Operators and Staff 

A one day workshop for training board and care providers was developed and pilot tested during 
Phase I of this project The field test of the workshop was conducted at eight sites throughout the 
state of Maryland. Approximately 263 providers attended the workshops. Of those 144 complete 
sets of data were obtained and used in the data analysis. Results of that analysis showed that 
participants' mean score improved from pre- to post test an average of 24%. As a group, they 
improved at least slightly on all test items. Results of the course evaluation suggest a strong 
approval by participants and a believe that they had gained knowledge of fire safety. 

Information collected by observed showed that instructors had generally followed the program as it 
was intended to be presented. Due to lack of time, however, in most workshops instructors omitted 
certain sections. The omitted sections varied from instructor to instructor. Instructors tended to 
spend more time than had been allotted on the modules covering fire behavior and life safety 
standards, and less time on all other modules. Module 7 which presented suggestions for the 
providers to conduct staff and resident training in their residences was not taught in any of the 
field test site... 

Conclusions and Recommendations 

The current study has taken a major step towards the development of a uniform, validated fire 
safety training system for board and care operators. The goal of the Instructor's Workshop, to 
train participants to present the Provider's Workshop in a uniform manner was at least partially 
fulfilled. Field test results indicate a broad, positive acceptance of the workshops. 

The dilemma, however, of reducing the scope of the Provider's Workshop so that it could be covered 
in one day and at the same time include all of the information that providers need to know 
remains to be resolved. Data from all data sources (instructors, participants, observers) strongly 
suggest that the amount of information presented is overwhelming to providers and instructors. 

Recommendations 

The following list presents our principal conclusions as regards to changes needed in the program 
for Phase III: 

1, During the pilot test of the Instructor's Workshop, a strong emphasis was placed on the 
content (fire safety and its implications for people with developmental disabilities). More 
emphasis needs to be placed on presenting the material in an interactive format-c.g., instructors 
must ask questions, encourage participants to be active learners. To achieve that goal, BWA will 
revise the format of the Instructor's Workshop to allow time for participants to practice teaching 
portions of the Provider's Workshop, and will present the Provider's Workshop in one day, 
modeling the method that it should be presented to providers. Also, to achieve that goal, BWA will 
add short QUIZZES at appropriate points in the Provider's Coursebook and other response type 
activities. BWA also will develop Guidelines for Presenting Workshop including training 
techniques and criteria for selecting a site. This material will become part of the Instructor's 
Workshop training and a resource for the instructors in the field. 
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2. The consensus of all data sources suggests that there was too much information to be 
covered in a single day. Not only was there not enough time to present all of the materials, but 
participants were "overwhelmed" with the amount of technical information they tried to absorb. 
BWA proposes to respond to that problem both through revising the Coursebook and by improving 
the training in the Instructor's Workshop. BWA asserts that, if the material were presented as 
described in the Instructor's Manual, it could be covered in the time allotted without overwhelming 
the participants. Although the information presented in the Participant's Coursebook was 
important and much of the material will be useful as a reference after the workshop, BWA will 
reduce the length of the Coursebook by simplifying the information in the module on fire behavior, 
and by removing pages of the Coursebook that were not covered in any workshop and which appear 
not to serve a strong purpose in presenting the major objectives. 

3. The training aids produced for the Provider's Workshop which every instructor received 
as part of the training package included: The Instructor's Manual, the Provider's Coursebook, two 
videos (Need for Fire Safety, and Flashover: Point of No Return), and an audiocassette (Human 
Behavior in Fires.) Information from all data sources suggests that these aids were acceptable 
and adequate to present the content. All sources agreed that the Provider's Coursebook should be 
permanently bound rather than presented in loose leaf, 3 hole punch forma t. The training aid 
with the lowest approval was the audiocassette. In our view, it was the format, not the content, that 
contributed to the low rating. When future funds are available, this training aid could be 
converted to a videotape. Additional training aids will be made available through a Continuing 
Education Program. 

4. Data from all sources agreed strongly that the open-ended question format used for the pre- 
and post test was unsuitable for the Provider's Workshops due to unreliability and difficulty in 
administration. Revising these instruments to a multiple choice format could serve to improve 
reliability and to allow more time during the Provider's Workshop for presenting the content. 
BWA proposes to develop two forms of a multiple choice instrument that can be used for pre- and 
post testing. 

5. The major task ahead for BWA during Phase III of the project will be to gain acceptance of 
the system by licensing agencies, national associations for board and care providers, members of 
fire marshall offices and local fire departments, and others who would be in a position to present 
and require fire safety training. One method that will be pursued is to seek peer review by 
developing papers and presentations and seeking to find audiences in professional publications 
and conferences. 
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Chapter 1 
Design of the Study 



This chapter describes the study design and procedures that were followed. 

Background 

The long range goal of the project to develop a Fire Safety Certification System for Board and Care 
Operators is to establish a training certification system whereby operators and staff of board and 
care homes for residents with developmental disabilities throughout the United States can receive 
uniform, validated fire safety training. Project staff was assisted by members of an Advisory 
council that met in a focus group format to discuss project issues and make recommendations 
regarding system design, course content, administrative procedures, types of materials, and other 
topics. Focus Group members included experts in fire safety, developmental disabilities, board 
and care operations, and training. 

During Phase I of the project, the course content for the Board and Care Provider's Workshop was 
developed. A coursebook, training aids, and an instructor's manual were then produced and pilot 
tested with staff of Melwood Training Center, Nanjemoy, Maryland, a non-profit organization 
that operates several alternative living units and group homes for adults with developmental 
disabilities. Training aids included a video titled The Need for Fire Safety," an audiocassette 
titled "Human Behavior in Fires," and 14 job aids-checklists for providers to use to conduct 
inspections or record fire drills and other activities. The results of the pilot test were reported in 
the Phase I Final Report dated December 20, 198?. | ^ g ^ 

Also during Phase I, the system was designed to include four components: the fire safety training 
workshop for board and care operators and staff, a workshop to train certified instructors 
(individuals v. ho would led these workshops), a continuing education program, and a system 
network. 



During Phase II of the project, the workshop for instructor's was developed and pilot tested. Sixteen 
participants completed the two-day workshop including the requirement to conduct a workshop for 
board and care operators within 90 days of the training. In all, eight workshops were conducted 
with 249 providers in attendance. 
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Purposes of the Study 

The primary purposes of the study described in this report were: 

1. To determine how effective the workshops were at preparing instructors to lead the 
workshop for providers and at conveying information about fire safety to providers, 
and 

2. To collect information about the instructor's training program and the provider's 
workshop which would aid project staff in improving fiiture versions. 



Organization of the Report 



The design of the pilot test of the Instructor's Workshop is described in Chapter 2 of this report. 
Results from the pilot test are presented in Chapter 3. The design of the field test of the Provider's 
Workshop is described in Chapter 4 and the results in Chapter 5. Chapter 6 presents the results 
from a ninth workshop conducted after results from the field test had been analyzed. Trainers 
incorporated suggestions from a review of the results by the project's Advisory Council in a Focus 
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Group meeting on June 26, 1989. Additional information was also sought from the twelve 
instructors who had led the field test workshops. The results of the follow-up survey are presented 
in Chapter 7. 

The next five sections of this report present summaries of the findings, problems, and 
recommendations for changes to both workshops and the workshop materials. Chapter 8 
summarizes all of the findings, the problems identified, and possible solutions for future 
workshops or editions of materials. The next five sections, Chapters 9 through 13, present 
recommendations for improving the Instructor's Workshop, the Instructor's Manual, the 
Provider's Workshop, the Provider's course materials, and the Evaluation Instruments. 

The final chapter in this report is titled "Implications for Phase III" and describes the manner in 
which the recommendations will be implemented in future workshops, in future editions of course 
materials, and with regards to the development of new training aids. 



Chapter Z 

Instructor's Workshop Pilot Test - June 5-6, 1989 



Purpose of the Instructor's Workshop 

The primary purpose of the Instructor's Workshop was to prepare participants to lead workshops for 
board and care providers using the Participant's Coursebook, other training aids, and evaluation 
instruments. A second purpose was to provide these future instructors with additional information 
about fire safety, developmental disabilities, needs of the board and care owners and operators, 
and training techniques needed to effectively present the workshop. 

Recruitment of Trainers for the Instructor's Workshop 

Trainers to lead the Instructor's Workshop were chosen from the Project Advisory Council and 
BWA project staff. They were Dr. Bonnie L. Walker, Ms. April L. Walker, Mr. Harry Bradley, 
P.E., and Ms. Terezie Bohrer. They were chosen because of their expertise in the fields of fire 
safety, fire protection engineering, developmental disabilities, or educational training. A team 
teaching approach was used and each trainer was assigned modules to teach related to their field of 
expertise. Each of the trainers were familiar with the modules they were assigned to teach. In 
each case they had been closely involved in the development of the course content and in some 
cases in the writing of the course materials. The four trainers met for planning sessions to 
develop the agenda and presentation. 

Agenda 

The workshop took place over a two day period. The approach was to present the Fire Safety 
Workshop for Board and Care Operators and Staff to the participants as they would be expected to 
teach it with the addition of enrichment materials. Also included on the program were four mini- 
lessons about training techniques. (A copy of the agenda is attached as Appendix A.) 

Materials 

The pilot test version of the Instructor's Manual was 3 hole punched and packaged in a 3 ring 
binder. Each module was separated with oversized divider pages printed on lavender coverstock. 
The Instructor's Manual was 143 pages, printed on one side only. 

The manual consisted of two types of materials: 

1. A reduced version of the activities pages from the Participant's Coursebook printed in 
black ink with a statement of the activity objective, instructions for presenting the 
information in red ink at the top of the page. To the right of the reduced coursebook, also 
in red ink, were key points and interesting facts to further guide the instructors in 
presenting the materials. 

2. In addition to the activity pages, the instructor's manual also had "pink" pages which 
were summaries of the modules, a list of objectives, instructions for introducing the 
modules, response sheets to activities in the Coursebook, and additional information 
on various subjects. 

In addition to the manual, each instructor received a training aids kit which included two 
videotapes, an audiocassette, evaluation instruments (pre- and post test, course evaluation). Four 
additional training aids were used in the Instructor's Workshop: two 16mm films produced by the 
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National Fire Protection Association, one videotape developed under a grant from the Department 
of Defense, and one videotape distributed by the Tobacco Institute. (See Figure 2-1 for a description 
of all seven audiovisual training aids listed according to the module in which they were used.) 

Figure 2-1 

Audiovisual Aids Used in Instructor's Workshop 



Introduction Module The Need For Fire Safety 

The Need for Fire Safety is an eight minute video produced by Bonnie Walker and Associates for 
the National Fire Safety Certification System Workshops. The purpose of the video is to point 
out the need for fire safety in residential care homes and to demonstrate various fire safety 
devices that are essential for people with developmental disabilities. 

Module One Fire Behavior 

Flashover: Countdown to Disaster is a sixteen minute film produced by the National Fire 

Protection Association for general audiences. The purpose of the film is to demonstrate the 
path and speed in which smoke and flames can spread through a building. 

Flashover: Point of No Return is a twelve minute video produced by the National Bureau of 

Standards for the staff of nursing homes. The purpose of the video is to demonstrate how a fire 
can be contained and how fire deaths can be prevented through the use of tight doors and proper 
fire evacuation procedures. 

Module Three Human Factors in Fire Safety 

Silent Response is an eight minute video produced through funding from The Tobacco Institute for 
fire emergency personnel. The purpose of the video is to help people recognize and understand 
how a person with hearing impairments might react in an emergency situation. 

Human Behavior in Fires is a seven minute audiocassette produced by Bonnie Walker and 
Associates for the National Fire Safety Certification System Workshops. The purpose of the 
audiocassette is to point out the characteristics of human behavior in fires as determined by 
fire safety research and to examine how people's backgrotind and previous experiences will 
affect the way that they react to a fire emergency. 

Module Four Fire Hazards 

Everybody Needs a Buddy Sometime is a twelve minute video produced through funding by the 
Department of Defense for the general audiences. The purpose of the video is to demonstrate 
how the buddy system can be used to evacuate people with physical disabilities from highrise 
buildings during a fire emergency. 

Module Five Fire Safety Devices 

Know Your Fire Extinguisher is a fifteen minute film produced by the National Fire Protection 
Association. The purpose of the film is to explain the three different types of fire and the 
various types of fire extinguishers that may be used to put out those fires. 

Recruitment of Instructors 

The majority of the participants who attended the Instructor's Workshop (17 of 19) were case 
managers for the State of Maryland's Certified Adult Residential Environment Program (CARE), 
commonly referred to as Project Home. One participant was the Housing Specialist for Project 
Home. One participant was a fire safety trainer and a member of the Project's Advisory Counsel. 
Project Home was originally established to provide supervised housing and case management 
services to persons who had been discharged firom state mental hospitals. However, in 1978, the 
target population was broadened to include adults with developmental disabilities from the 
community. The program is operated by the Maryland Department of Human Resources and local 
Departments of Social Services. 



Setting for the Study 



The Instructor's Workshop took place at the offices of Bonnie Walker and Associates in Bowie, 
Maryland on June 5 and 6, 1989. The two-day training was conducted in a spacious training room 
in a round table lecture format The conference room was equipped with a projection screen, 
overhead projector, 16mm film proj^or, videotape player and monitor, an audiocassette player, 
chalkboard, and flipchart. Three BWA staff members were also present to videotape the workshop, 
assist in data collection, and observe the proceedings. 

Sample 

Eighteen Project Home employees and one fire safety specialist participated in the Instructor's 
Workshop. To qualify for the Instructor's Workshop, participants completed an application for 
Level II Training (the Certified Instructor's Program). Requirements for the workshop had been 
established by the Project Advisory Council. Each person was required to have at least an AA 
degree and to have prior experience in one or more of the following areas: developmental 
disabilities or other special needs population, residential care facilities or other health facilities, 
fire safety, or training. The participants who qualified for the June 5-6 workshop primarily had 
experience dealing with the certification and supervision of residential board and care homes. 
Table 2-1 below presents information on how the instructors met the criteria for selection. The 
information was obtained from pre-registration forms that were filled r ut by participants prior to 
the training. (See Appendix fl.) 



Table 2-1 

Criteria for Selection of Instructors (N=19) 



Requirement 


Number Meeting 
the Requirement 


Percentage 


Number Not 
Meeting the 
Requirement 


Percentage 


Education 


19 


100% 


0 


0 


Developmental 
Disabilities 


18 


94.8% 


1 


5.2% 


Residential Care 
Facilities 


18 


94.8% 


1 


5.2% 


Fire Safety 


9 


47.4% 


10 


52.6 


Training 


6 


31.5% 


13 


68.5% 



Data Collection and Analysis 

Preregistratiou Form 

To assess their qualifications for the Instructor's Workshop, each participant was asked to 
complete a preregistration form. Copies of the form were mailed to the Project Home Coordinator a 
few weeks before the training. Ten forms were received through the mail and nine additional 
forms were completed at the training site. 

The preregistration forms had been distributed by mail to the participants about three weeks prior 
to the training. Additional forms were distributed to members of the group who had not previously 
completed a form when they arrived at the workshop. The results were analyzed to confirm that the 
participants were qualified to be trained as instructors. 
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PreVPostTest 



Two versions of the workshop test, Form A and Form B, was developed to assess the cognitive 
learning of the participants. Each form contained 21 items. The item with the same number on 
each one tested the participants' knowledge of the same module and content domain. Test 
questions and standardized responses were reviewed for content validity by experts in fire safety, 
developmental disabilities, data analysis, and training. Table 2-2 below identifies the location in 
the course materials where the information required to answer each item could be found. 
Appendix A contains a copy of both forms of the test and the standardized answer key. 

The pre- and post test items had been modified from those used during the pilot test. In that version 
questions had been written completely open-ended such as: "Why is there a greater need for fire 
safety in residences for people with developmental disabilities?" Participants were given points 
for each response resulting in a wide range of scores for each item and greatly affecting the 
reliability of the scores. Further, since the same or similar items appeared on the pre- and post 
tests, some participants who had listed several responses on the pretest, listed fewer responses on 
the post test, thus resulting in lower scores. To correct the problem, the field test version of the test 
modified items to read: "List 3 reasons why there is a greater need for fire safety in residences for 
people with developmental disabilities." Some of the items were also modified slightly to improve 
clarity and readability. (See Appendix C for the Scoring Guide to Form A and B.) 



Table 2-2 

Match of Test Form A and B Questions and Course Materials 



Item Number 


Form A 


FormB 


1 


Introduction, p. 3 


Introduction, p. 3 


2 


Module 2, p. 10, Module 4, p. 15 


Module 4, p. 16 


3 


Module 1, p. 12 


Module 1, p. 11 


4 


Module 2, p. 7 


Module 2, p. 7 


5 


Module 6, p. 4 


Module 6, p. 4 


6 


Module 2, p. 6 


Module 2, p. 6 


7 


Module 2, p 10 


Module 2, p. 9 


8 


Module 3, p. 1 


Module 3, p. 3 


9 


Module 3, p. 4 


Module 3, p. 4-12 


10 


Module 3, p. 5-12 


Module 3, p. 15 (IM*), 6, p. 4 


11 


Module 3, p. 14 


Module 3, p. 14 


12 


Module 4, p. 9 


Module 4, p 7 


13 


Module 4, p. 4-5, (IM) 


Module 4, p. 1 


14 


Module 5, p. 6 


Module 5, p. 2 


15 


Module 5, p. 14 


Module 5, p. 14 


16 


Module 5, p. 10 


Module 5, p. 10 


17 


Module 5, p. 1 (IM) 


Module 6,p.4 


18 


Module 6, p. 7 


Module 6, p. 8 


19 


Module 6, p. 9 


Module 6, p. 10 


20 


Module 6, p. 3 


Module 6, p. 3 


21 


Module 2, p. 5 


Module 2, p. 5 



* Instructor's Manual 



Participants received Form B as a pretest and Form A as a post test. The tests were scored based on 
a standardized answer key that had been reviewed for content validity by experts in fire safety, 
developmental disabilities, data analysis, and training. Each item was assigned a maximum 
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point value based on the number of responses required. The maximum score on both forms of the 
test was 50. 

Data were analyzed to determine the mean score for the total group and the mean score on each 
item. Data were also analyzed by item to determine whether or not a majority of the participants 
had improved their scores on a majority of items. 



Course Evaluation Form 

A fifteen item Likert-type evaluation instrument was developed to assess the participants' 
perceptions of their own learning. In addition, the participants were asked to suggest ways to 
improve the Instructors Workshop. (See Appendix D.) 

The participants' responses to each item were tabulated, converted to percentages, and presented by 
item and response category. 

Figure 2-2 

Match of Modules and Items on the Course Evaluation Form 



Module 


Item 


The Need for Fire Safety 


1,8 


Fire Behavior 


none 


Life Safety Standards 


7 


Human Factors in Fire Safety 


4,5,13 


Fire Hazards 




Fire Safety Devices 


none 


Fire Emergency Planning 


2 


Staff and Resident Training 


none 


Workshop 


3, 6,9, 10, 11,12 



Training Aids Evaluation Forms 

Following the training, the participants completed an evaluation of the seven audiovisual 
training aids used in the workshop. Each of the training aids was rated on a scale of 1 to 5, with 1 
as poor and 5 as excellent (See Appendix E.) 

The participants' responses to each item were tabulated, converted to percentages, and presented by 
item and response category. 



Training Observation 

During the training, one person observed and recorded observations including the time each 
module began and ended. The observer was instructed to note any questions participants asked, 
information provided by instructors not in the manual, and other events. In addition, a videotape 
was made of the entire two day training. The videotape was used to record and analyze participant 
questions and to provide suggestions for future improvements to both the coursebook and the 
instructor's manual. A transcription of the videotape was prepared and analyzed by project staff. 

Information obtained from the observer's notes and the videotapes was compiled and analyzed. 
Questions asked by the participants and trainers that were not in the instructor's manual or 
coursebook were noted, as well as pertinent information related to the teaching of each module, 
such as the time required to teach each module and the success of the teaching technique Used. The 
results of the analysis are presented as Tables 3-7, 3-8 and 3-9 in Chapter 3. 




Chanter 3 

Presentation of the I nstructor's Workshop Results 



Introduction 

The findings of this study represent an analysis of the data obtained from nineteen participants 
who participated in the pilot test of the instructor's workshop held at the offices of Bonnie Walker 
and Associates, Bowie, Maryland on June 5 and 6, 1989. All of the participants except two were case 
managers for Project Home. One participant was the Housing Specialist for Project Home, and 
the other was a fire safety training expert who is also a member of the project's Advisory Counsel. 
Data were obtained from pre- and post tests, the course evaluation, the training aids evaluation, 
from observations by an observer, and from a transcript of the videotape of the training. 

Pre- and Post Test Results 

Question One: Did the participants, as a group, improve their scores from pre- to post test? 

Scores of each participant who completed both a pre- and post test were totaled. Descriptive data for 
both the pretest and post test were calculated using StatView™, a computer program for the 
.Macintosh published by BrainPower Inc., 1986. Results indicate that participants' mean score 
improved from 19.75 on the pretest to 38.625 on the post test for a mean gain score of approximately 
19 points. Results can be seen in Table 3-1 below. 

Table 3-1 

Presentation of Descriptive Data for the Pretest and the Post Test (N»16) 



Xi; Pretest 



Mean: 


Std. Dev.: 


Std. Error: Variance: Coef. Var.: Count: 


19.75 


5.36 


1.34 


28.733 


27.141 


16 


Minimum: 


Maximum: 


Ranqe: 


Sum: 


Sum Squared: # Missing: 


1 1 


30 


19 


316 


6672 


0 


Mean: 


Std. Dev.: 


X2: Post Test 

Std. Error: Variance: Coef. Var.: Count: 


38.625 


3.263 


.816 


10.65 


8.449 


16 


Minimum: Maximum: 


Ranqe: Sum: 


Sum Squared: # Missinq: 


32 


44 


12 


618 


24030 


0 



To test the significance of the difference, a simple regression analysis was performed between the 
pretest mean (dependent Y variable) and the post test mean (the independent X variable.) The 
results of that analysis are presented in Table 3-2 below. 
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Table 3-2 

Comparison of Pre- and Post Test Scores Using Regression Analysis (N=16) 



Simple Regression X*| : Pretest Y*| : Post Test 



DF: 


R: 




R-squared: 


Adj. R-squared: 


Std. Error: 


Il5 


.28 




.078 


.013 


3.243 






Analysis of Variance Table 




Source 


DF: 


Sum Squares: 


Mean Square: 


F-test: 


REGRESSION 


1 


12.534 


12.534 


1.192 


RESIDUAL 


14 


147.216 


10.515 


p = .2934 


TOTAL 


1 5 


159.75 







No Residual Statistics Compute 



Simple Regression Xi : Pretest Y-) : Post Test 



Beta Coefficient Table 



Parameter: 


Value: 


Std. Err.: 


Std. Value: 


t-Value: 


Probability: 


INTERCEPT 


2.367 










SUOPE 


1.353 


.3 


.718 


4.503 


.0002 



Confidence Intervals Table 



Parameter: 


95% Lower: 95% Upper: 90% Lower: 90% Upper: 


I MEAN (X,Y) 


17.321 


25.155 


18.002 


24.474 


SUDPE 


.724 


1.981 


.833 


1.872 



Figures 3-1 and 3-2 below depict the relationship between the instructors' pretest and post test scores 
in a scattergram and then in histograms. 
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Figure 3-1 

Scattergram showing Correlation Between Instructors' Pre- and Post Test Scores (N=16) 

y s .lYlx + 35.257, R-squared: .078 
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Figure 3-2 

Histograms showing distribution of Scores on Instructors' Pre- and Post Tests (N=16) 



Histogram of X-| : Pretest 
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Histogram of X2: Post Test 
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Pre- and Post Test Results by Item 

Question Two: Did the participants improve their scores from pre- to post test on each item? 

The scores of each participant on each item of the pre- and post test were compared to deterr ane 
how many of the participants showed an improvement between the pre- and post test on each item of 
the test. Results showed that they had improved at least slightly and often dramatically on all but 
three items. (See Table 3-3 below.) On Items 8, 9, and 12, the participants' mean scores were 
slightly less on the post test than on the pretest . 



Table 3-3 

Comparison of Pre- and Post Test Results by Item (N=16) 



Item 


Module 


Maximum 
Points 


Pretest 
Average 


Post Test 
Average 


Difference 


Result 


1 


1 


3 


1.125 


1.875 


+.75 


Improvement 


2 


4 


5 


2.5 


4.625 


+2.125 


Improvement 


3 


1 


2 


0 


1.438 


+1.438 


Improvement 


4 


2 


2 


.938 


1.438 


+.50 


Improvement 


5 


6 


2 


.875 


1 OCT 

1.25 


+.375 


Improvement 


6 


2 


2 


.188 


2 


+1.81 


Improvement 


7 


2 


3 


2 


2.69 


+.69 


Improvement 


8 


3 


2 


1.75 


.563 


-1.19 


Negative Result 


9 


3 


2 


1.25 


.813 


-.438 


Negative Result 


10 


3 


2 


.625 


2 


+1.375 


Improvement 


11 


3 


3 


1 


2.125 


+1.125 


Improvement 


12 


4 


1 


1 


.938 


-.063 


Negative Result 


13 


4 


4 


.313 


2.94 


+2.62 


Improvement 


14 


5 


2 


.688 


1.94 


+1.25 


Improvement 


15 


5 


1 


.438 


1 


+.562 


Improvement 


16 


5 


1 


.313 


.875 


+.562 


Improvement 


17 


5,6 


2 


.063 


1.25 


+.62 


Improvement 


18 


6 


2 


1.38 


1.94 


+.558 


Improvement 


19 


6 


3 


.375 


2.06 


+1.69 


Improvement 


20 


6 


3 


2.188 


2.88 


+.687 


Improvement 


21 


2 


3 


.75 


1.94 


+1.19 


Improvement 


Total 






19.759 


38.580 


+18.236 




Ave. 






.940 


1.84 


+.87 






Individual and Group Improvement by Item 



Question Three: Did the participants improve from pre- to post test on individual items? 

The data were also analyzed to compare the number of items on which each individual improved from pre- 
to post test. Results presented in Table 3-4 below show that a majority of the participants (15 out of the 16) 
showed improvement on a majority of the items (11 or more). On the average, participants improved or 
stayed the same on 19.25 of the 21 items. 

Characteristics of the testing instrument and test environment were examined to determine problems 
which could have contributed to test unreliability particularly on items where participants' scores had 
decreased from pretest to post test. Factors noted were: (1) The items on Form A and Form B were not 
equal in terms of difficulty. (2) A fatigue factor influenced post test results since participants had spent 
two full days, and eight hours that day, in the workshop. Many of the participants had driven a substantial 
distance to the workshop site and faced a lengthy drive home after the workshop was dismissed. (3) In 
some cases, the sections of the module which covered the information tested on the items which had been 
omitted by the workshop trainers. (4) In some cases, the participants already scored the maximum 
number of points on the pretest and could not improve. (5) In some cases the participants may have been 
out of the room while a particular topic was being covered. 



Table 3-4 

Comparison of Performance by Individuals from Pre- to Post Test 
by Workshop <N«16) 



Participant 


Improved 


Same 


Negative 


Total 


1 


11 


8 


2 


21 


2 


13 


7 


1 


21 


3 


14 


6 


1 


21 


4 


12 


7 


2 


21 


5 


8 


9 


4 


21 


6 


17 


3 


1 


21 


7 


13 


6 


2 


21 


8 


14 


5 


2 


21 


9 


12 


8 


1 


21 


10 


13 


7 


1 


21 


11 


15 


4 


2 


21 


12 


14 


4 


3 


21 


13 


12 


6 


3 


21 


14 


11 


9 


1 


21 


15 


15 


6 


0 


21 


16 


14 


5 


2 


21 


Total 


208 


100 


28 




Mean 


13.0 


625 


1.75 




Median 


13 


&5 


2 




Mode 


12 


5,6,73 


2 




Range 


8-17 


3-9 


04 





ERIC 



24 

13 



Affective Evaluation Results 

The participants (after having completed the post test) also completed the Fire Safety Instructor's 
Workshop Evaluation Form, a Likert-Scale type instrument designed to measure their feeling about the 
content of the training they had just received and their ability to administer the training to care providers 
themselves. A copy of the form is included as Appendix D. 

Results of the Course Evaluation 

Question Four: Did the participants have positive feelings about the workshop and their learning? 

The evaluation included eight items that measured participants' feeungs about the cognitive content of the 
workshops. The remaining seven items measured feelings about the. presentation of the workshop. 

The results of the analysis presented below in Table 3-5 suggest an overall approval of the 
participants of the workshop and a feeling that they believed they had gained in knowledge during 
the two day workshop. Approximately 86% of the participants either agreed or agreed strongly with 
each statement. Since all the statements were positive statements about the training, that was the 
desired response. Item 8 which dealt with the importance of fire safety training for residential 
care providers and Item 9 which dealt with the hazardous consequences of smoking received the 
most positive rating. Item 3 which dealt with the participants' having a clear idea of how to present 
the workshop material to providers and Item 7 which dealt with licensing and certification 
standards received the least number of positive ratings. 
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Table 3-5 

Participants' Attitudes Towards the Training and Their Learning Experience (N-17) 



Items 


Strongly 
Agree 


Agree 


Uncertain 


Disagree 


Strongly 
Disagree 


1. I was motivated by this workshop to 
learn more about fire safety. 


53% 


41% 


0% 


6% 


0% 


2. I am able to identify and do something 
about fire safety issues. 


35% 


59% 


6% 


0% 


0% 


3. I have a clear idea about how to present 
the workshop materials to providers. 


0% 


41% 


53% 


0% 


6% 


4. I am comfortable with the terminology 
related to residential care facilities that 
I need to know to present the workshop. 


12% 


76% 


6% 


0% 


6% 


5. I am comfortable with the terminology 
related to developmental disabilities 
that I need to know to present the 
workshop. 


23% 


71% 


6% 


0% 


0% 


6. I am comfortable with the terminology 
related to fire safety that I need to know 
to present the workshop. 


6% 


70% 


18% 


0% 


6% 


7. I have a clear understanding about 
licensing and/or certification 
requirements related to fire safety. 


6% 


35% 


53% 


0% 


0% 


8. Fire safety instruction for residential 
care providers is important. 


94% 


6% 


0% 


0% 


0% 


9. I have a clear understanding of the 
consequences of smoking and other 
high risk behaviors. 


94% 


6% 


0% 


0% 


0% 


10. 1 was encourage during the workshop to 
ask questions, and to express my ideas 
and feelings. 


53% 


47% 


0% 


0% 


0% 


ll.The instructors of this workshop were 
well prepared and were knowledgeable 
of the subject matter they presented. 


41% 


59% 


0% 


0% 


0% 


12. 1 would recommend this workshop to 
others. 


41% 


53% 


6% 


0% 


0% 


13. I have a clear understanding of the day- 
to-day responsibilities and tasks of 
residential care providers. 


35% 


59% 


0% 


0% 


6% 


14. The instruction and practice with 

respect to training that I received in this 
workshop was helpful. 


24% 


59% 


12% 


0% 


0% 


15. The training aids that the presenters 
used during this workshop were 
appropriate and helped me learn. 


29% 


71% 


0% 


0% 


0% 


Averages (rounded) 


36% 


50% 


11% 


<1% 


2% 
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Results of the Training Aid Evaluation 



Question Five: Did the participants feel that the training aids were useful and appropriate? 

Seven different audiovisual aids were used during the Instructor's Workshop. At the conclusion 
of the workshop, participants were asked to fill out a Training Aid Evaluation Form and to rate 
each aid on a scale of 1 to 5, with 1 being poor and 5 being excellent. The results are presented in 
Table 3-6 below. The participants rated Flashover: Countdown to Disaster the highest, with 88% 
rating it as excellent Flashover: Countdown to Disaster, a public domain video produced by the 
National Bureau of Standards, received the second highest rating. Know Your Fire Extinguisher 
was rated the lowest, with 18% of the participants giving it a poor rating. 



Table 3-6 

Participants' Attitudes Towards Training Aids Used in the Workshop (N-17) 



Title 


Excellent 


Good 


Average 


Below 
Average 


Poor 


No 

Response 


Flashover: 
Countdown to 
Disaster 


88% 


6% 


0% 


6% 


0% 


0% 


Flashover: 
Point of No 
Return 


76% 


18% 


0% 


6% 


0% 


0% 


The Need for 
Fire Safety 


29% 


47% 


18% 


0% 


0% 


6% 


Human 
Behavior in 
Fire 


18% 


41% 


29% 


12% 


0% 


0% 


Silent 
Response 


12% 


41% 


35% 


6% 


0% 


6% 


Everybody 
Needs a 
Buddy 
Sometime 


18% 


23% 


47% 


0% 


6% 


6% 


Know Your 
Fire 

Extinguisher 


12% 


18% 


23% 


29% 


18% 


0% 


Total 


40% 


29% 


22% 


4% 


1% 


3% 
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Observation Data 

£2XX£E? &SZ^£%^~S*~*+* * not s ,a rt a» H , 9 . M a m 
make up time . Module 0nc „„ rM^ZSm mta " g ™? Part 1 Coined to ' 

workshop ended on time at 4:30 p.m * SS " me than had be «" allotted. The 

™ Table 3-7 

Time Expenditure for Training by Module in Minutes 



120 



Introducti on 
Pretest 
Overview 

Training Tips. Par* i 

introduction to Participant's" I 36" 
Workshop 1 

I Module l.Fire Behavin g 
I Lunch 

moauie ggg S afety" Standard I Sf 

Module 3, Human FactorTfa 

Fire Safety 

Training Tips. P ar f -9 , ~ 

UayTwo ~ ' 

I Training Tips, Part 3 ' 
Module 4. Fire Hazards 
Module 5. Fi re Safety Dev i^T 

I Lunch 

[Module 6, Fire Emergency" 
Planning 

Module 7, Resident Staff" 
Training 

Continuing Education 



Difference 



30 



85 



15 



15 



23 



-20 



-19 



0 



-35 
"-15~ 



+8 



I 30 



15 



-27 
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Question Seven: 



How did the scope of the content vary from the planned presentation? 



A transcript was prepared from the videotape recording of the workshop and analyzed to determine 
the additional information given by the trainers and questions posed to trainers by the participants 
that was outside of the scope. A summary of those findings are presented below by module in 
Tables 3-8 and 3-9. 



Table 3-8 

Unplanned Information Provided by Trainers 



Module 


Comments 


Introduction 


• The issue of fires caused by arson was raised. 

• Legal liability was discussed. 


Fire Behavior 


•Woodstoves were briefly discussed. 

• The Triangle of Fire vs. The Tetrahedron of the Fire was 
explained. 

• Techniques and equipment used bv firefighters was discussed. 


Life Safety Standards 


• Additional handouts summarizing Chapters 17, 20, and 21 of 
NFPA s Life Safety Codes were distributed. 


Human Factors in Fire 
Safety 


rai uiipdjii/o were a&KcQ ii you nao to cnoose co nave a 
disability, which would you choose? Which would you like to have 
least?" 


Fire Hazards 


• The dangers of frying food with oil was discussed. 

• The flow of electricity into appliances and the definition of 
voltage was discussed. 

• The insulation of fireplace chimneys was explained. 

• The hazards of deadbolts was discussed. 


Fire Safety Devices 


• The proper location of smoke detectors on the wall and ceiling 
was pointed out. 

• The advantages of hard- wired smoke detectors versus battery- 
operated detectors was discussed. 


Fire Emergency Planning 


None 


Staff and Resident 
Training 


None 


CEP 


None 
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Table 3-9 

Unanticipated Participant Questions During Instructor's Workshop 



Module 


Questions 


Introduction 


•What are the risks from arson? 


Fire Behavior 


wny won t a wooastove set on the smoke 

ucicttur wxiiic xjuniiiig toast Will* 

•How can firefighters enter a room that is 
extremely hot without getting burned? 
•What should you do if you have a gas leak? 
•If you shut the doors and windows, won't the 
room explode? 


Life Safety Standards 


•Can the staff be sued if they are not doing all 
they should concerning fire safety? 
•Why are exit signs required above doors in 
residential homes? 


Human Factors in Fire Safety 


None 


Fire Hazards 


•What do the numbers on the chart "Risk of 
Jjeatn mean : (Module 4, page 1) 
•Can you use blankets or baking soda to put out 
kitchen fires? 

•Does bending cords cause damage? 

•Are multi-outlet strips better than extension 

Luruo » 

•What should the clearance for my woodstove 
be? 

•What 1^ fchP 1 flrtnfll 1 a w rnnrprnintr rlna^Knlf 

locks? 

•What should I do about wandering clients? 


Fire Safety Devices 


•Where should smoke detectors be mounted? 
On the wall? On the ceiling? 
•Do all multiple station detectors have to have 
the same alerting mechanism, or can some 
have strobe lights while others have buzzers? 


Fire Emergency Planning 


•Do I have to have structural changes done to 
my home to make an escape route from the 
second floor? 


Staff and Resident Training 


None 


CEP 


None 



Summary of Results by Module-Instructor's Workshop 

Introduction Module- The Need for Fire Safety 

The content from the Introduction Module was tested by Item 1 on the pre- and post tests, which dealt 
with the need for fire safety in residential care homes. A majority of the participants (10 out of 16) 
improved on this item. Five of the participants showed no change from pre- to post test, while one 
participant received a lower score the second time around. 

Items 1 and 8 on the Course Evaluation measured participants' affective response to the 
Introduction Module. Results shows that 94% of the participants felt motivated by the training to 
learn more about fire safety and 100% of the participants agreed that fire safety instruction for 
residential care providers is important. 




The video shown in the Introduction Module, The Need for Fire Safety \ was rated above average or 
excellent as a training aid by 76% of the participants. 

Module One-Fire Behavior 

The content in Module One was tested by Item 3 on the pre- and post tests. Item 3 on the pretest 
asked the participants to define the term flashover. The matching item on the post test asked them 
to identify the danger of smoke inhalation. A majority of participants improved their score from 
pre- to post on Item 3 (13 out of 16). The remaining three participants showed no change in their 
score. 

No items were included on the course evaluation to assess the affective response of the participants 
towards the information in the Fire Behavior module. 

The video Flashover: Countdown to Disaster, produced by the National Fire Protection 
Association, which was shown during Module One, received the highest rating of all the training 
aids. It received an excellent rating by 88% of the participants. Flashover: Point of No Return, 
produced by the National Bureau of Standards, also shown in Module One, received the second 
highest rating with 76% of the participants rating it as excellent. 

Module Two- life Safety Standards 

Items 4, 6, 7, and 21 on the pre- and post test measured content covered in Module Two, Life Safety 
Standards. The participants, as a group, showed on improvement from pre- to post test on all of 
these items. 

Item 6, which was related to vertical openings, showed the largest gain average. A low score on the 
pretest item and a high gain indicated that most of the participants didn't understand the term 
vertical openings prior to the workshop, but understood the term after the workshop. 

Item 21 asked participants to name the three components of a safe egress, also showed a strong 
gain. The majority of participants (10 out of 16) improved on this item from pre- to post test. 

Fire safety code requirements (post test) and hazardous areas (pretest) were covered by Item 7. On 
these items, 9 out of 16 participants improved, 6 stayed the same, and one showed negative 
improvement. Lack of improvement could be attributed to the post test item being more difficult 
than the pretest item. 

Item 4 tested participants' knowledge of interior finishes. Participants knew what interior 
finishes were and that they should be fire resistant, but only a few participants mentioned class 
ratings which was the desired response. Class ratings and fire-resistant interior finishes will be 
addressed further in the Continuing Education Program materials. 

Participants' responses to Item 7 on the Instructors Workshop Evaluation Form suggest that they 
were still somewhat unclear concerning licensing and/or certification requirements related to 
fire safety. 

Module Three- Human Factors in Fire Safety 

Items 8, 9, 10, and 11 measured cognitive learning related to client and staff characteristics, 
community resources, and human behavior in fires. 

Participants showed positive improvement on both Item 10, dealing with the physical limitations of 
people with developmental disabilities, and Item 11, dealing with human behavior in fires. 



A negative improvement was exhibited on Items 8 and 9. These negative responses can be 
attributed to two factors. First, Form B of the test which was administered as the post test was more 
difficult than Form A of the test which was administered as the pretest for Items 8 and 9. Second, 
the information needed to answer these items were not stressed during the workshop. 

Affective responses to Module Three were measured on Item 5 of the Instructor's Workshop 
Evaluation Form. Of the participants, 94% expressed that they felt comfortable with terminology 
related to developmental disabilities. 

The video Silent Response distributed by the Tobacco Institute was shown during Module Three. 
Although 53% of the participants felt that the video was above average, the majority also felt that the 
video was not suitable for the workshop because it was developed for fire emergency personnel. 
The audiocassette, Human Behavior in Fires, was rated above average or excellent by 59% of the 
participants. 



Module Four-Fire Hazards 



Module Four was measured by responses to Items 2, 12, and 13. On Items 12 and 13, participants 
showed a large positive gain. On Item 2, 100% of the participants improved from pre- to post test, 
and on Item 13, 94% of the participants improved. Thus, participants' knowledge of fire hazards 
appeared to dramatically increase during the workshop. 

Item 12 showed a slight negative improvement which was because all 16 participants knew the 
correct pretest answer, while all but one knew the correct post test answer. 

Item 9 on the Instructor's Workshop Evaluation Form measured participants' affective responses 
to information about fire hazards. All of the participants either agreed or strongly agreed that they 
had a clear understanding of the consequences of smoking with respect to fire safety. 

The video Everybody Needs a Buddy, a program about evacuation of people with handicaps from a 
high-rise building in a fire emergency was produced under a grant from the Defense Department. 
It was rated as above average or excellent by 88% of the participants. 

Module Five-Fire Safety Devices 

Knowledge about fire safety devices was measured on the pre- and post test by Items 14 thorough 16. 
Items tested participants 1 knowledge of smoke alarms for people with hearing impairments, fire 
extinguishers, and sprinkler systems. Participants showed a positive gain on all of these items 
even though less time than was planned was spent on this module. The largest large gain was 
shown in the item measuring knowledge about smoke alarms for people with hearing 
impairments. 

There were no items on the Instructors Workshop Evaluation Form to measure affective 
responses to Module Five. The film, Know Your Fire Extinguisher, produced by the National Fire 
Protection Association, was shown during the Fire Safety Devices module. Only 30% of the 
participants rated the film as above average or excellent. This video was produced in the 1950's 
and some of the content was dated. 



Finally, Items 5 and 17 through 20 dealt with topics related to fire emergency planning. 
Participants showed improvement from pre- to post test on all items, with a particularly large gain 
on Item 19 dealing with the frequency of fire drills. Other item topics included the requirements 
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for choosing alternate evacuation routes, the hazards of fire ladders, tasks for the staff and 
residents in a fire emergency, and ways of improving fire safety in a home. 

Item 2 on the Instructor's Workshop Evaluation Form measured the affective responses of 
participant's to Module 6. Over 94% of the participants agreed or strongly agreed that after the 
workshop they could identify and do something about fire safety issues and problems. 

Module Seven-Resident and Staff Training 

There were no pre- or post test or affective evaluation items to measure the response to 
Module Seven. 



ERIC aa 3 3 



Chapter 4 

Field Test of Provider's Workshop - June 12-21, 1989 



Setting for the Study 

The field test of the Fire Safety Training for Board and Care Operators took place at eight sites 
throughout the state of Maryland. Project Home care providers participated in one day workshops. 
Project Home is a state program operated by departments of social services throughout Maryland. 
This program provides Certified Adult Residential Environment (CARE) homes where disabled 
persons 18 years of age and older are accepted, cared for, and supervised. This program serves 
individuals who are mentally ill (41%), elderly (32%), developmental^ disabled (25%), diabetic 
(1%), and homeless (1%). CARE homes are usually private homes which accept the placement of 
disabled persons through Project Home. As of November 1988 there were 365 care providers, 404 
approved Project Home sites, and 929 certified beds. 
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Sample 

Two hundred and forty-nine people participated in the field test of the Fire Safety Training for 
Board and Care Operators. Of the total number of participants, 144 people completed both the pre- 
and post test, 176 completed the post test, 45 people completed only the pretest and 29 completed 
neither a pre- or a post test. Only 144 out of the 151 people completing both the pre- and post test were 
considered in the data analysis. Six of the participants mailed in their post test too late to be 
included in the analysis. One married couple completed the pretest individually and the post test 
together, making their scores ineligible for the analysis. Only those participants who completed 
both a pretest and a post test were included in the data analysis measuring differences from pre- to 
post test 



Figure 4-1 
Workshop Participants 



Number completing pre-and post test 


151* 


Number completing pretest and no post test 


45 


Number completing post test and no pretest 


24 


Number who attended but completed no tests 


29 


Total number attending 


249 



* 6 participants mailed in the post test after the workshop. A few couples took the pretest separately 
and the post test together. Those scores are not included in the data analysis measuring 
differences between pre- and post tests. 

The majority of the participants were care providers from Project Home with one to three clients 
living with them in residential homes or apartments. A few participants were Project Home case 
workers. Tables 4.1 to 4.7 below present information about the participants collected on the 
Preregistration Form (See Appendix B). These data can be used to develop a general profile of the 
participants. 
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Table 4-1 

Distribution of Workshop Participants by Experience With Disabilities (N»139) 





Total 




utcilbai U XOCLUlllltj 


iXJO 


76% 


Mental Retardation 


62 


45% 


Mobility Impaired 


34 


24% 


Hearing Impaired 


20 


14% 


Visual Impairments 


19 


14% 


Epilepsy 


19 


14% 


Other 


15 


11% 


Autism 


8 


6% 


Cerebral Palsy 


7 


5% 


The percentages do not add up to 100% because participants responded to more than one item. 



Table 4J2 

Distribution of Workshop Participants by Years Experience with People with Disabilities (N»139) 



Years 


0-1 Years 


1-5 Years 


6-10 Years 


11+ Years 


NR 


Total 


8 


65 


34 


27 


5 


Average 


6% 


47% 


24% 


19% 


4% 



Table 4^ 

Distribution of Workshop Participants by Areas of Special Training (N»139) 





Total ~ 


Percentage 


Nutrition 


62 


45% 


Medication 


49 


35% 


Management 


45 


32% 


Developmental Disabilities 


44 


32% 


Rehabilitation 


37 


27% 


No Response 


30 


22% 


Education 


24 


17% 


Recreation 


23 


17% 


*Other 


22 


16% 



* Other Responses included Social Work, Personal Care, Intensive Behavior, Special Olympi 
Psychiatric Nursing, Nursing, Vocational Training, Housing, and Counselor. 
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Table 4.4 

Distribution of Workshop Participants by Education Level Attained (N-139) 



Education Level 


Total 


Percentage 


High School 


100 


72% 


Elementary School 


12 


9% 


A.A. Degree 


9 


7% 


Bachelor's Degree 


8 


6% 


Master's Degree 


6 


4% 


Doctorate 


2 


1% 


No Response 


2 


1% 



Table 45 

Distribution of Workshop Participants by Years at the Facility (N=139) 



Years 


0-1 Years 


1-5 Years 


6-10 Years 


11+ Years 


NR 


Total 


18 


72 


33 


7 


9 


Percentage 


13% 


52% 


24% 


5% 


6% 
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Table 4.6 

Prior Fire Safety Training of Workshop Participants (N=139) 



Previous Training 


Yes% 


No% 


No Response 


Previous Fire Safety Training 


43% 


55% 


2% 


Fire Safety Training at the Same Facility 


6% 


94% 


0% 


Experience Training other Staff 


29% 


40% 


31% 


Experience Training Clients 


61% 


19% 


19% 


Experience with a Fire Emergency* 


14% 


73% 


13% 



♦Responses include: Fire in 1980, minor injuries and damages. Careless smoking by client while 
sitting in a chair. The fire was smoldering and water was put on the fire. A child was playing 
with matches and started a fire. 



Table 4-7 

Length of Previous Fire Safety Training (N= 60) 



Length 


One Hour 


2-4 ours 


1/2 Day 


One Day 


One day + 


No 

Response 


Total 


8 


20 


1 


20 


3 


8 



9 
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Data Collection and Analysis 

Preregistration Form 

To collect information regarding the demographic characteristics of the field test group, each 
participant was asked to complete a Preregistration Form.(See Appendix B.) Copies of the form 
were mailed to supervisory personnel at Project Home a few weeks before the training. Additional 
copies of the form were distributed at the training. 
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Preregistration forms were distributed to participants about three weeks prior to the training. 
BWA received ninety-six preregistration forms from individuals prior to the training. Forms 
from forty-six additional providers were collected at the training sites. Preregistration data were 
analyzed only for those participants who completed the workshop, N=139. Five participants 
completed the workshop but did not complete a preregistration form. (See Tables 3.1 through 3.7.) 

Pre-/PostTest 

A pre-post test was developed to assess the participants' knowledge of fire safety issues before and 
after the workshop. The test questions were reviewed for validity by experts in fire safety, 
developmental disabilities, data analysis, and training. Two versions of the test, Form A and 
Form B, were developed with 21 items each. Each item with the same number on each form of the 
test corresponded to the same module and tested the same knowledge domain. See Chapter 2, page 
14, Table 2-2, Match of Form A/B Questions and Module which identifies the module and page 
number in the Coursebook that contains the information requested in each item. 

In order to determine whether Forms A and B were comparable, participants in Workshops 1,4, 6, 
and 8 were assigned Form A as a pretest and Form B as a post test. In Workshops 2, 3, 5, and 7 
participants received Form B as the pretest and Form A as the post test. Analysis of the results 
showed that Form B of the test was slightly more difficult than Form A. The group that took Form B 
as their pretest and Form A as their post test showed a slightly higher gain than those that took A as 
their pre-test and B as their post test. (Results are presented on page 26.) 

The tests were scored based on a standardized answer key that has been reviewed by experts in fire 
safety, developmental disabilities, data analysis, and training. The maximum possible score was 
50.(Appendix C is the answer key for Forms A and B.) 

The data from the 144 participants who completed both a pre- and post tests were analyzed by 
individual, by item, and by workshop. The data were also analyzed by Group A/B and Group B/A 



Course Evaluation Form 

A Likert-type evaluation instrument was developed to assess the participants' affective learnings. 
The evaluation contained fourteen items to assess the participants' perception of their own 
learning. (See Appendix D for a copy of the form.) Participants were instructed not to put their 
names on this form. 



After completing the workshop, participants were asked to fill out a course evaluation. The 
participants' responses to each item on the evaluation were totaled and converted to percentages to 
determine whether they believed they had learned the information. These results were tabulated 
and presented by item and response. 

Provider Survey 

A questionnaire containing sixteen items was developed to better understand the needs and 
environment of residential care facilities. The survey items were designed gain information 
on the types of facility that the participants operate, the types of fire safety devices that the facilities 
contain, and the characteristics of the residents that live in the facilities. (See Appendix F for a 
copy of the form.) Participants were instructed not to put their names on this form. 

After completing the workshop, participants were asked to fill out a survey. The results of the 
survey were compiled and presented in tabular form. 
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Training Observation 



During each workshop one representative of BWA observed and recorded observations. The 
observer noted the length of time spent on each module, the person who taught the module, and what 
pages were covered or skipped. In addition, the observer recorded responses to the training and 
variations from the planned structure using the Instructor's Manual. 

Information from the eight observers was combined and analyzed so that all of the unexpected 
events were noted. Those events included questions the trainers or participants asked that were not 
covered by the training material; questions or information that were not asked or covered that 
were in the training materials; and interruptions in the training. The information per module 
was compared with items on the post test on which participants scored poorly. Information from the 
eight observers was compiled and summarized and used to make recommendations for changes 
in the training materials and the Instructor's Workshop. 
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Chapter 5 

Presentation of the Provider's Field Test Results 



The findings of this study represent an analysis of the data obtained from the 144 participants who 
completed both the pre- and post tests in eight one-day fire safety workshops held at various sites in 
the state of Maryland during the period from June 12, 1989 to June 21, 1989. 



Pre- and Post Test Results 

Question One: Did the participants, as a group, improve their scores from pre- to post test? 

The scores of each participant on the pre- and post test were totaled. Description data were compiled 
using StatView™, a computer statistical analysis published by BrainPower, Inc. in 1986. The 
results indicate that the participants' mean score improved from 12.424 points on the pretest to 
24.368 on the post test, a mean gain score of approximately 12 points. Those results are presented 
in Table 5-1 below. 



Table 5.1 

Presentation of Providers' Descriptive Data for the Pretest and Post Test (N=144) 



X-j: Pretest 



Mean: 


Std. Dev.: 


Std. Error: Variance: Coef. Var.: Count: 


12.424 


6.438 


.537 


41 .449 


51.821 


144 | 


Minimum: 


Maximum: 


Range: Sum: 


Sum Squared: 


# Missinq: 


|l 


30 


29 


1789 


28153 


I 


Mean: 


X2: Post Test 

Std. Dev.: Std. Error: Variance: Coef. Var.: 


Count: 


|24.368 


8.963 


.747 


80.332 


36.781 


144 ] 


Minimum: Maximum: 


Range: Sum: Sum Squared: # Missina: 


0 1 4- 1 


41 


3509 


96995 jo J 



To test the difference between the pre- and post test average scores for statistical significance a 
simple regression test was performed on the data, shown in Table 5.2, which indicated that the 
higher one had scored on the pretest, the higher that individual scored on the post test. In other 
words, individuals gained in proportion to their original knowledge. Results showed that the 
difference was statistically significant at the probability level of .0001. 




Table 5-2 

A Regression Analysis of the Differences Between Pre- and Post Test Scores (N=144) 



Simple Regression Xi : Pretest Y-| : Post Test 



DF: 


R: 


R-rquared: 


Adj. R-squared: 


Std. Error: 


15 


.28 




.078 


.013 


3.243 


Source 


DF: 


Analysis of Variance T 
Sum Squares: 


able 

Mean Square: 


F-test: 


REGRESSION 


1 


12.534 


12.534 


1.192 


RESIDUAL 


1 4 


147.216 


10.515 


p - .2934 


TOTAL 


1 5 


159.75 







No Residual Statistics Computed 



Simple Regression : Pretest Yi : Post Test 

Beta Coefficient Table 



3 arameter: 


Value: 


Std. Err.: 


Std. Value: 


t-Value: 


Probability: 


INTERCEPT 


2.367 










SLOPE 


1.353 


.3 


.718 


4.503 


.0002 



Confidence Intervals Table 



Parameter: 



MEAN (X.Y) 


17.321 


25.155 


18.002 


24.474 


SLOPE 


.724 


1.981 


.833 


1.872 
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Figure 5-1, below, illustrates the difference between the group mean scores in a bar chart. 



Figure 5-1 (N-144) 

Bar Chart of Column Moans: X-j ... X2 
26 1 1 j 

24. J 1 

22. 

2 20. 

*c 

Z> 

18. 
16. 
14- 

12J 1 ; i I ( I 

Pretest Posttest 
Columns 

Pre- and Post Test Results by Item 

Question Two; Did the participants improve their scores from pre-to post test on each item? 

The scores of each participant for each item of the pre- and post test were compared to determine 
how many of the participants showed an improvement between pre- and post test on each item of the 
test Results presented in Table 5-3 below show that as a group the participants had improved at 
least slightly on all but Item 5, which measured participants ability to state characteristics of 
alternate escape routes or to identify the names of the two kinds of escape routes— primary and 
alternate. As a group the average improvement per item was .57, approximately half a point per 
item. 
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Table 5.3 

Comparison of Pre- and Post Test Results by Item (N=144) 



Item 


Module 


lVffiTimiiTn 

Points 


Average 


Pn«t Tent 

Average 






i 


l 


3 


1.00 


1.11 


+.11 


Improvement 


2 


4 


5 


2.29 


3.17 


+.88 


Improvement 


3 


1 


2 


.36 


1.01 


+.65 


Improvement 


4 


2 


2 


.73 


.93 


+.2 


Improvement 


5 


6 


2 


.49 


.39 


-.1 


Negative Result 


6 


2 


2 


.15 


1.08 


+.93 


Improvement 


7 


2 


3 


1.43 


2.17 


+.74 


Improvement 


8 


3 


2 


.57 


.67 


+.1 


Improvement 


9 


3 


2 


.38 


.71 


+.33 


Improvement 


10 


3 


2 


.47 


1.00 


+.53 


Improvement 


11 


3 


3 


.40 


.93 


+.53 


Improvement 


12 


4 


1 


.62 


.92 


+.3 


Improvement 


33 


4 


4 


.41 


1.55 


+1.14 


Improvement 


14 


5 


2 


.53 


1.33 


+.8 


Improvement 


15 


5 


1 


.26 


.56 


+.3 


Improvement 


16 


5 


1 


.15 


.46 


+.31 


Improvement 


17 


5,6 


2 


.16 


.54 


+.38 


Improvement 


18 


6 


2 


.63 


1.52 


+.89 


Improvement 


19 


6 


3 


.31 


1.28 


+.97 


Improvement 


20 


6 


3 


.69 


1.87 


+1.18 


Improvement 


21 


2 


3 


.40 


1.15 


+.75 


Improvement 


Total 




50 


12.43 


24.35 


+11.92 




Ave. 






.59 


1.16 


+.56 





Individual and Group Improvement by Item 

Question Three: Did the participants improve from pre- to post test on individual items? 

The data were also analyzed by workshop to determine the number of items on which each 
individual improved from pre- to post test. Results showed that 66 out of 144 participants (46%) had 
improved on a majority of items (11 or more.) Table 5-4 shows the average number of items on 
which each participant had improved , stayed the same , or showed a negative improvement for all 
eight workshops. The average participant improved on ten items, showed no change on nine items, 
and showed negative improvement on two items. 
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Table 5-4 

Comparison of Performance by Individuals from Pre- to Post Test by Workshop (N=8) 



Workshop 


N 


Average 
Improved 


Averacfe 
Same 


Averacfe With 

Negative 

Improvement 


Total Nnmhpr 
of Items 


1 


2 


8.5 


11.0 


1.5 


21 


2 


7 


9.1 


8.4 


3.4 


21 


3 


21 


10.1 


8.0 


2.9 


21 


4 


18 


8.8 


9.2 


3.1 


21 


5 


34 


11.1 


8.1 


1.8 


21 


6 


20 


9.8 


8.6 


2.7 


21 


7 


25 


10.5 


8.0 


2.6 


21 


8 


17 


8.6 


10.1 


2.3 


21 


Total 


144 


9.93 


8.56 


2.51 




Average 












% 




47% 


41% 


12% 





Question Four: Did the form of the test taken (Form A or Form B) affect how the participants 

improved from pre- to post test? 

Participants in Workshops L 4, 6 and 8 took Form A as the pretest and Form B as the post test. 
Participants in Workshops 2, 3, 5, and 7 took Form B as the pretest and Form A as the post test. An 
analysis was performed to compare the two groups regarding individual performance on each 
item from pre- to post test. Table 5-5 presents the findings of that analysis. The AB group improved 
on an average of 9.1 items. The BA group improved on an average of 10.5 items, 1.4 more items 
than the AB group. 



Table 5,5 

Comparison of Performance by Individuals from Pre- to Post Test By Post Test Group (N=*144) 



Group 


Number of 


Average Items 


Average Items 


Average Items 


Total Number 




Participants 


Improved 


the Same 


Negative 


of Items 










Improvement 




AB 


N=57 


9.1 


9.3 


2.6 


21 


BA 


N=87 


10.5 


8.1 


2.4 


21 


Difference 


Mean 


+1.4 


-1.2 


+.2 





Affective Evaluation Results 

The participants (after having completed the post test) also completed a Course Evaluation Form 
(See Appendix A), a Likert-Scale type instrument designed to measure their feelings about the 
training they had just received. Participants were instructed to circle the response which best 
described their feelings about a list of 15 statements. On this measurement, 157 participants 
completed the evaluation. 



The results of the analysis presented below in Table 5-6 suggest an overall approval by the 
participants and a belief that they had gained knowledge of fire safety during the workshop. On the 
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average, approximately 90 percent of the participants either agreed or strongly agreed with each 
statement. Since all items were positive statements about the training, that was the desired 
response. Items 2 and 3 which dealt with identifying fire safety problems and making their 
facilities safer received the most positive ratings. Item 14 which dealt with the Continuing 
Education Program received the least positive ratings, with 59% of the participants agreeing or 
strongly agreeing that the program would be useful to them. 

Table 5-6 

Participants' Attitudes Towards the Training and Their Timing Experience (N«157) 



Items 


Strongly 
Agree 


Agree 


Un- 
certain 


Disagree 


Strongly 
Disagree 


NR* 


1. I was motivated tc learn morp about firp 
safety by this training session. 


43% 


47% 


1% 

kJ rU 




1 °L 


AO/- 


2. I am better able to identify and do 
something about fire safety problems than I 
was before this training. 


48% 


48% 


1% 


1% 


1% 


1% 


3. I have a better understanding of the 
things I must know about to make my home 
and clients safer from fire. 


51% 


47% 


1% 


0% 


1% 


0% 


4. Issues were raised which caused me to 
rethink the current fire emergency 
procedures in my home. 


41% 


49% 


4% 


3% 


1% 


1% 


5. I have a better understanding of the 
special needs and risks of people with 
developmental disabilities. 


39% 


54% 


2% 


2% 


2% 


1% 


6. I have a better understanding of the 
strengths and limitations of staff. 


29% 


61% 


4% 


1% 


1% 


4% 


7. I have a clearer understanding of the 
licensing requirements for residential 
care facilities than I did before. 


34% 


55% 


6% 


2% 


1% 


2% 


8. I have a better understanding of the 
things I must do regularly to prevent a fire. 


41% 


54% 


1% 


1% 


1% 


2% 


9. I am more aware of the consequences of 
smoking and other high risk behaviors. 


44% 


48% 


1% 


4% 


1% 


2% 


10. 1 was encouraged to express my thoughts 
and feelings in this workshop 


29% 


61% 


3% 


3% 


1% 


3% 


11. The instructors were well-prepared to 
conduct this workshop. 


44% 


45% 


5% 


4% 


0% 


2% 


12. 1 will use the Coursebook after the 
workshop as a fire safety reference. 


46% 


48% 


2% 


0% 


1% 


3% 


13. The job aids (checklist) will help me 
with my fire safety program. 


44% 


49% 


2% 


1% 


0% 


4% 


14 The Continuing Education Program will 
be useful to me and I plan to subscribe. 


26% 


33% 


26% 


2% 


1% 


12% 


Averages (rounded) 


40% 


50% 


4% 


2% 


1% 


3% 



*NR=No response 
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33 



44 



Observation Findings 

Question Six: How did the workshops vary from the plan in terms of the time spent on each topic? 

The data presented in Table 5-7 below show that the largest block of time was spent on Module 1, Fire Behavi 
each workshop on Module 1, which was 12 minutes more than the allotted time. The least amount of time was 
the average, three minutes was spent on Module 7 (12 minutes less than the allotted time) and one minute w; 
Education Program (14 minutes less than the allotted time.) Instructors spent 38 minutes on the average tea 
than the allotted time. Although all but two of the eight workshops started late, all but two ended on time or ei 



Table 5-7 

Time Expenditure at Each Workshop for Training by Module in Minutes (N«8) 
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ST 


PT 


I 


1 


B 


2 


3 


B 


4 


5 


B 


6 


7 


CEP 


PT 


End 


1 


6/12 


-20 


15 


22 


SB 


18 


46 


61 


18 


36 


11 


10 


35 


0 


0 


30 


0 


2 


6/13 


-15 


27 


24 


59 


13 


0 


40 


25 


26 


17 


5 


54 


2 


1 


ND 


ND 


3 


6/14 


-15 


25 


18 


53 


10 


19 


36 


24 


31 


33 


0 


35 


9 


5 


19 


+15 


4 


6/15 


0 


50 


26 


59 


20 


25 


45 


15 


20 


20 


0 


32 


0 


0 


20 


0 


5 


6/16 


-12 


22 


24 


35 


15 


39 


31 


41 


36 


35 


0 


29 


5 


0 


40 


-15 


6 


6/19 


-15 


22 


25 


43 


11 


39 


22 


23 


30 


30 


0 


40 


10 


5 


45 


0 


7 


6/20 


-11 


19 


30 


60 


15 


35 


25 


30 


30 


30 


0 


30 


0 


0 


ND 


ND 


8 


6/21 


0 


20 


20 


52 


18 


45 


15 


20 


20 


15 


0 


45 


0 


0 


40 


-10 


T 




-88 


200 


189 


419 


120 


239 


275 


1S6 


229 


191 


15 


300 


26 


11 


194 


-10 


M 




-11 


25 


24 


52 


15 


30 


34 


25 


29 


24 


2 


38 


3 


1 


32 


-2 


AT 






30 


30 


40 


5 


30 


30 


15 


30 


25 


5 


60 


15 


15 


30 




D 






-5 


-6 


+12 


+10 


0 


+4 


+10 


-1 


-1 


-3 


-22 


-12 


-14 







KEY to Table 

ST-Starting Time, Numbers indicate how many minutes late the workshop started. 
PT-Pretest or Poet test 

I-Introductory Module, includes showing Need for Fire Safety video 

1-7, Module Numbers 

B-Break 

CEP — Continuing Education Program 

End-Numbers indicate how many minutes late the workshop ended 
ND-No data were collected by the observer. 
M-Mean (Numerical average) 
AT— Allotted time 

D — Difference between allotted time and time taken 




Question Seven: How did the workshops vary from the plan in terms of the coverage of each topic? 



Table 5-8 

Content Covered by Each Workshop by Module 
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Field Test Summary 



Introduction Module— The Need for Fire Safety 

The content from the Introduction Module was tested by Item 1 on the pre- and post test. Results of 
the data analysis presented in Table 5-8 below show the overall score improvement for Item 1 was 
positive (+4%), evidence that there was a gain in knowledge concerning the need for fire safety. 
Item 1 on the Course Evaluation which measured the participants 1 affective response to the 
Introduction Module suggests that participants felt that they were motivated by the training to learn 
more about fire safety. Ninety percent of the participants either agreed or strongly agreed with that 
item. 



Table 5-9 

An Analysis erf Pre- and Post Test Results for the Introduction Module 



Item 


N 


Content 


Poss. 
Point 


Ave 

Score 

Pretest 


Ave 
Score 
Post Test 


%on 
Pretest 


%on 
Post Test 


Result 


1 


144 


The Need for Fire Safety 


3 


1.00 


1.11 


33% 


37% 


+4% 



Module One — Fire Behavior 



The content from Module One was tested by Item 3 on the pre- and post test. The results , suggest an 
overall gain in knowledge on the item concerning fire behavior. There was no Course Evaluation 
item to measure the participants' affective response concerning fire behavior. 



Table 5.10 

An Analysis of Pre- and Post Test Results for Module One 



Item 


N 


Content 


Poss. 
Point 


Ave 

Scare 

Pretest 


Ave 
Score 
Post Test 


%on 
Pretest 


%on 
Post 
Test 


Result 


3 


144 


Fire Behavior 


2 


.36 


1.01 


18% 


51% 


+33% 



Module Two-Life Safety Standards 



The content from Module Two was tested by Items 4, 6, 7, and 21 on the pre- and post test. The 
overall results for Module Two suggests that the participants gained in knowledge concerning 
Life Safety Standards. Item 6 concerning vertical openings showed the largest gain (+47%). Item 
4, which dealt with interior finishes, showed the least gain score (+10%). Participants also showed 
improvement on Item 21 (components of a safe egress), with a gain in score of +25% and Item 7 
(code requirements and hazardous areas) with a gain score of +24%. 

Participants' affective response to the information in Module Two was measured by Items 2 and 7 
on the Course Evaluation. Over 96% of the participants either agreed or strongly agreed with Item 2 
concerning identifying and correcting fire safety problems. Over 89% of the participants either 
agreed or strongly agreed, Item 7 which stated that participants had a clearer understanding of the 
licensing requirements for residential care faciliH.es. 
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Table 5-11 

An Analysis of Pre- and Post Test Results for Module Two 



Item 


N 


Content 


Poss. 
Point 


Ave 

Score 

Pretest 


Ave 
Score 
Post Test 


%on 
Pretest 


%on 
Post 
Test 


Result 


4 


144 


Interior Finishes 


2 


.73 


.93 


37% 


47% 


+10% 


6 


144 


Vertical Openings 


2 


.15 


1.08 


7% 


54% 


+47% 


7 


144 


Code Requirements and 
Hazardous Areas 


3 


1.43 


2.14 


48% 


72% 


+24% 


21 


144 


Components of a Safe 
Egress 


3 


.40 


1.15 


13% 


38% 


+25% 



Module Three-Human Factors in Fire Safety 

The content from Module Three was tested by Items 8, 9, 10, and 11. The overall result (positive 
improvement on every item) suggests that the participants gained in knowledge concerning 
Human Factors in Fire Safety. Items 10 and 11 showed the largest improvement in score, +26 % on 
Item 10 and +18% on Item 11. 

Characteristics of direct care staff and resident activities that impact on fire safety (Item 8) showed 
the least improvement (+5%) from pre- to post test. Item 9, which dealt with characteristics of people 
with developmental disabilities showed an improvement in score of +17% from pre- to post test. 

Items 5 and 6 on the Course Evaluation measured the participants' affective responses to Module 
Two. Response to these items showed that 93% of the participants either agreed or strongly agreed 
that they had a better understanding of the special needs and risks of people with developmental 
disabilities; 90% of the participants agreed they had a better understanding of the strengths and 
limitations of staff. 



Table 5-12 

An Analysis of Pre- and Post Test Results for Module Three 



Item 


N 


Content 


Poss. 
Point 


Ave 

Score 

Pretest 


Ave 
Scare 
Post Test 


%on 
Pretest 


%on 
Post 
Test 


Result 


8 


144 


Characteristics of Staff 
and Residents 


2 


.57 


.67 


29% 


34% 


+5% 


9 


144 


Characteristics of People 
with Developmental 
Disabilities 


2 


.38 


.71 


19% 


36% 


+17% 


10 


144 


Physical Limitations 
and Community 
Resources 


2 


.47 


1.0 


24% 


50% 


+26% 


11 


144 


Human Behavior in 
Fires 


3 


.40 


.93 


13% 


31% 


+18% 
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Module Four-Fire Hazards 



The content in Module Four was tested by Items 2, 12, and 13. Participants showed the largest gain 
in score (+30%) on Item 12, which dealt with the risk of electrical and appliance fires. Item 13, 
which dealt with the risk of cooking and smoking fires also showed a large gain in score (+29%). 
Participants showed the least gain in score (+17%) on Item 2, which dealt with the components of 
monthly and bedtime fire safety inspections. 

Item 10 on the Course Evaluation measured participants' affective responses to Fire Hazards. 
Responses to this item showed that 92% of the participants either agreed or strongly agreed that they 
had a clearer understanding of the hazards of smoking and other high risk behaviors. 



Table 5-13 

An Analysis of Pre* and Post Test Results for Module Four 



Item 


N 


Content 


P088. 

Point 


Ave 

Score 

Pretest 


Ave 
Scare 
Post Test 


%on 
Pretest 


%on 
Post 
Test 


Result 


2 


144 


Fire Safety Inspections 


5 


2.29 


3.17 


46% 


63% 


+17% 


12 


144 


Risk of Appliance and 
Electrical Fires 


1 


.62 


.92 


62% 


92% 


+30% 


13 


144 


Risk of Cooking and 
Smoking Fires 


4 


.41 


1.55 


10% 


39% 


+29% 



Module Five-Fire Safety Devices 

Knowledge about fire safety devices was measured on the pre- and post test by Items 14 through 16. 
Item 14 concerning smoke detectors showed the largest improvement in score (+40%). Item 16, 
which dealt with sprinkler systems, showed an improvement in score of +31%. The least 
improvement in score (+30%) was shown on Item 15 dealing with fire extinguishers. 

There were no items to measure the affective response of participants to Module Five. However, at 
the end of Module Five, in five of the ei^.t workshops, Quick Draw was played, a game which 
involved participants drawing fire safety devices on the chalkboard while other participants 
guessed what the items were. 



Table 5-14 

An Analysis of Pre- and Post Test Results for Module Five 



Item 


N 


Content 


Poss. 
Point 


Ave 

Scare 

Pretest 


Ave 
Score 
Post Test 


%on 
Pretest 


%on 
Post 
Test 


Result 


14 


144 


Smoke Detectors and 
Alarms 


2 


.53 


1.33 


27% 


67% 


+40% 


15 


144 


When to use a fire 
extinguisher 


1 


.26 


.56 


26% 


56% 


+30% 


16 


144 


Knowledge of sprinkler 
systems 


1 


.15 


.46 


15% 


46% 


+31% 
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Module Six-Fire Emergency Planning 



Items 5, 17, 18, 19 and 20 dealt with topics related to fire emergency planning. Participants showed 
positive improvement on Items 17, 18, 19 and 20. Item 18 showed the largest improvement in score 
(+44%). Item 20 showed an improvement of +39%, Item 19 showed an improvement of +33%, and 
Item 17 showed an improvement of +19%. 

Negative results (-5%) were shown on Item 5, which dealt with the criteria for choosing alternate 
escape routes and the number of escape routes required from each bedroom. 

Affective responses to Module Six were measured by Items 2 and 4 on the Course Evaluation. 
Responses to these items showed that 96% of the participants either agreed or strongly agreed that 
they were better able to identify and do something about fire safety problems than before the 
training; 90% of the participants either agreed or strongly agreed that issues were raised which 
caused them to re-think the current emergency procedures in their home. 



Table 5-15 

An Analysis of Pre- and Post Test Results for Module S ix 



Item 


N 


Content 


PO88. 

Point 


Ave 

Score 

Pretest 


Ave 
Scare 
Post Test 


%on 
Pretest 


%on 
l.'bst 
Test 


Result 


5 


144 


Alternate Escape Routes 


2 


.49 


.39 


25% 


j;o% 


-5% 


17 


144 


Fire Ladders and 
Primary Escape Routes 


2 


.16 


.54 


8% 


27% 


+19% 


18 


144 


Emergency tasks for 
staff and resident 


2 


.63 


1.52 


32% 


76% 


+44% 


19 


144 


Frequency and types of 
fire drills 


3 


.31 


1.28 


10% 


43% 


+33% 


20 


144 


Ways to improve fire 
safety 


3 


.69 


1.87 


23% 


62% 


+39% 



Module Seven— SYoJT? aad ^^dant Trcuo\n 



No data were collected about participant content regarding the learning in Module S 
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Chapter g 
Follow-Up Workshop Results 



Introduction 

Following the analysis of the data from the Instructor's Workshop and field test of the Providers 
Workshops, members of the project's Advisory Council met on June 25, 1989 to discuss the results of 
the evaluation including their own observations. All but one member of the Advisory Council had 
either taught portions of the Instructor's Workshop, observed one or more of the Provider's 
Workshops, or taught one or more sessions of the Provider's Workshop. Results were collated 
regarding the presentation by instructors and other aspects of the workshop from all of the data 
sources, presented to the Council, and discussed. A set of preliminary recommendations were 
developed. To test whether those recommendations would have an important impact on the 
workshop outcomes as measured by the post test, BWA conducted another workshop on Saturday, 
August 5, 1989 using one BWA staff member and a member of the Focus Group who had completed 
the Instructor's Workshop on June 5 and 6, 1989. 

Specific recommendations that the BWA trainers were expected to implement were: 

• The number of participants should be limited to 25. 

• Participants should be allowed to use the coursebook to complete the post test. 

• The instructions in the Instructor's Manual should be followed more closely. 

• Participant interaction should be encouraged. Activities in the coursebook which 
ask for participants to write in the book should be emphasized. 

Setting 

The Follow-Up Workshop took place at the Reisterstown Road Library on Saturday, August 5, 1989. 
The training took place in a conference room equipped with tables, chairs, a television, a videotape 
player, an audiocassette player, and a chalkboard. 

Sample 

Twenty-five Project Home care providers participated in the Follow-Up Workshop. Of those, 
twenty-one people completed the pre- and post test, two others completed the pretest and no post test, 
and two did not complete either the pre- or post test. Twenty-one participants completed the 
Provider's Survey and eighteen completed the Course Evaluation. 



Figure 6-1 
Follow-Up Workshop Participants 



Number completing pre-and post test 


21 


Number completing pretest and no post test 


2 


Number who attended but completed no tests 


2 


Total number attending 


25 



A preregistration form was completed by 24 participants. Of those participants, 92% operated out of 
private residential homes. Over 67% of the participants worked with people with mental illness 
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and 42% had worked with people with disabilities for between one and five years. Approximately 
38% of the participants had had special training in nutrition and medications. Over 54% had 
completed high school, 13% had completed a bachelor's degree, and 13% had completed a master's 
degree. The remaining 20 % had completed less than a high school education. Previous fire safety 
training had been administered to 38% of the participants, and 46% had never trained their clients 
in fire safety procedures. Over 67% had never experienced a fire emergency. 

Data Collection and Analysis 

Participants completed a preregistration form, Form A as a pretest, Form B as a post test, a course 
evaluation, and a provider's survey. Pre- and post tests were scored using the identical procedures 
used in the field test. Data were collected from the course evaluation and provider's survey forms. 
The twenty-one participants that completed a pre- and a post test were included in the data 
analysis. Pre- and post tests were scored using the standardized answer key which listed all 
acceptable responses to the open-ended items. Each participant was assigned a number and the 
results of their tests were tabulated c or their total score and by item. Total scores and scores on 
each item were compared from pre- to post test. 

Findings 

pre-a-^c* Post Te^-t ^xls^W^ 

Question One: Did the participants improve from pre- to post test? 

For the 21 board and care providers attending the Follow-Up workshop, the mean score on the post 
test was 13.9 (28%) and the mean score on the post test was 21.2 (42.5%), a mean gain of 7.3 (10.6%) 
points. Complete results of the analysis for pre- and post tests are presented below in Table 6-1. 

Table 6-1 

Presentation of Follow-Up Providers' Descriptive Data far the Pretest and Post Test (N=21) 



X-j: Pretest 



Mean: 


Std. Dev.: 


Std. Error: Variance: Coef. Var.: Count: 


13.952 


6.383 


1.393 


40.748 


45.751 


21 


Minimum: 


Maximum: 


Range: 


Sum: Sum Sauared: # Missina: 


3 


27 


24 


293 


4903 


0 


Mean: 


Std. Dev.: 


X2: Post Test 

Std. Error: Variance: Coef. Var.: 


Count: 


21.238 


12.016 


2.622 


144.39 


56.579 


21 


Minimum: 


Maximum: 


Range: 


Sum: Sum Sauared: 


# Missinq: 


3 


43 


40 


446 


12360 


1 



A simple regression analysis was performed to determine whether the mean group gain of 
approximately 10 percentage points from pre- to post test was statistically significant. The results 
shown in Table 6-2 below show that gain was significant at the probability level of .0002. 
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Table 6-2 

Regression Analysis for Pre- to Post Test Results (N»21) 



Simple Regression Xf : Pretest Yi : Post Test 



DF: 


R: 




R-squared: 


Adj^ R-squared: 


Std. Error: 


[20 


.718 


.516 


.491 


8.575 


Source 


DF: 


Analysis of Variance Table 

Sum Squares: Mean Sauare: 


F-test: 


REGRESSION 


1 


1490.797 


1490.797 


20.276 


RESIDUAL 


19 


1397.012 


73.527 


p = .0002 


TOTAL 


20 


2887.81 







No Residual Statistics Computed 



Simple Regression X1 : Pretest Y*| : Post Test 

Beta Coefficient Table 



Parameter: 


Value: 


Std. Err.: 


Std. Value: 


t-Value: 


Probability: 


INTERCEPT 


2.367 










SLOPE 


1.353 


.3 


.718 


4.503 


.0002 



Confidence Intervals Table 



MEAN (X,Y) 


17.321 


25.155 


18.002 


24.474 


SLOPE 


.724 


1.981 


.833 


1.872 



I o 

LERIC 



54 

43 



Figure 6-2 

Histogram of Pretest and Post Test Results (N«21 ) 
Histogram of Xi : Pretest 
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Post Test 

Course Evaluation Results 

Question Two: Did the participants have positive feelings about the workshop and their 

learning experience? 

The results of the analysis presented below in Table 6-3 suggest an overall approval of the 
participants of the workshop. Over 87% of the participants either agreed or strongly agreed vith 
each positive statement about the workshop. Items 11 and 12 received the most positive ratings, with 
100% of the participants either agreeing or strongly agreeing with those items. Item 11 dealt with 
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the preparation of the instructors, while Item 12 dealt with using the Coursebook as a reference after 
the workshop. Item 14, which dealt with the Continuing Education Program, received the least 
positive rating, with 61% of the participants either agreeing or strongly agreeing with that item. 



Table 6-3 

Participants' Attitudes Towards the Training and Their Learning Experience (N-16) 



Itoms 


Agree 


AgI*<N5 


Un. 

KJ MM, 

certain 


1 lion rfrn n 


strongly 
Disagree 




1. I was motivated to learn more about fire 
safety by this training session. 


56% 


22% 


0% 


16% 


0% 


6% 


2. I am better able to identify and do 
something about fire safety problems than I 
was before this training. 


50% 


50% 


0% 


0% 


0% 


0% 


3. I have a better understanding of the 
things I must know about to make my home 
and clients safer from fire. 


44% 


56% 


0% 


0% 


0% 


0% 


4. Issues were raised which caused me to 
rethink the current fire emergency 
procedures in my home. 


44% 


44% 


0% 


11% 


0% 


0% 


5. I have a better understanding of the 
special needs and risks of people with 
developmental disabilities. 


22% 


61% 


0% 


16% 


0% 


0% 


6. I have a better understanding of the 
strengths and limitations of staff. 


28% 


50% 


0% 


11% 


0% 


11% 


7. I have a clearer understanding of the 
licensing requirements for residential 
care facilities than I did before. 


39% 


44% 


6% 


6% 


0% 


6% 


8. I have a better understanding of the 
things I must do regularly to prevent a fire. 


33% 


61% 


0% 


6% 


0% 


0% 


9. I am more aware of the consequences of 
smoking and other high risk behaviors. 


44% 


39% 


0% 


16% 


0% 


0% 


10. I was encouraged to express my 
thoughts and feelings in this workshop. 


39% 


50% 


0% 


6% 


0% 


6% 


11. The instructors were well-prepared to 
conduct this workshop. 


56% 


44% 


0% 


0% 


0% 


0% 


12. I will use the Coursebook after the 
workshop as a fire safety reference. 


56% 


44% 


0% 


0% 


0% 


0% 


13. The job aids (checklist) will help me 
with my fire safety program. 


44% 


39% 


6% 


0% 


0% 


11% 


14. The Continuing Education Program 
will be useful to me and I plan to subscribe. 


22% 


39% 


11% 


11% 


0% 


16% 


Averages (rounded) 


41% 


46% 


2% 


7% 


0% 


4% 




Question Three: How did the workshop vary from the plan in terms of time spent on each topic? 

Instructors arrived 22 minutes late. However, the Project Home Director had already started 
showing the video, The Need for Fire Safety. Participants took the Pretest after they had seen the 
video. For the rest of the workshop, the time taken did not substantially deviate from the time 
allotted. (See Table 6-4 below.) 



Table &4 

Time Expenditure for Follow-Up Workshop by Module in Minutes 



Module 


Time Allotted 


Time Used 


Difference 


Introduction to Participants 
Workshop 


30 


20 


-10 


Module 1, The Need for Fire 
Safety 


30 


ND 




Module 2, Fire Behavior 


40 


33 


-7 


5 minute break 


5 


20 


+15 


Module 3, Life Safety 
Standards 


30 


17 


-13 


Module 4, Human Factors 
in Fire Safety 


30 


28 


-2 


Break 


15 


25 


+10 


Module 5, Fire Hazards 


30 


35 


+5 


Module 6, Fire Safety 
Devices 


25 


30 


+5 


5 minute break 


5 


0 


-5 


Module 7, Fire Emergency 
Planning 


60 


60 


0 


Module 8, Staff and 
Resident Training 


15 


5 


-10 


Continuing Education 
Program 


15 


5 


-10 


Post Test/ Course 
Evaluation 


30 


40 


+10 



5 i 



46 



/ 



Question Four: How did the participants' performance in the Follow-Up Workshop compare 

with the performance of the participants in the Field Test Workshops? 

A comparison of gain between Field Test participants and Follow-Up Workshop participants 
showed that overall Field Test participants had improved about 5% more. Follow-Up participants, 
however had a slightly higher post test average. 

Tabled 

A Comparison of Pie-Post Test Results Among Field Test, Follow-up, and Instructor Groups 



Group 


N 


Pretest 


SD 


Percentage 


Post Test 


Percentage 


Difference 


SD 


Field Test 


144 


12.4 


6.44 


24.8% 


24.3 


48.6% 


33.8% 


8.96 


Follow-up 


19 


16.7 


6.38 


33.7% 


25.4 


50.8% 


17.5% 


12.02 


Instructors 


18 


19.75 


5.36 


39.5% 


38.6 


77.2% 


1S.8 


3.263 



Table &€ 

A Comparison of Pre and Post Test Results Among Individual Workshops (N«9) 



Workshop 


N 


Pretest 


Percentage 


Post Test 


Percentage 


Difference 


1 


2 


| 5.5 


11% 


13 


26% 


15% 


2 


7 


18 


36% 


2&4 


56.8% 


20.8% 


3 


21 


16 


32% 


27.9 


55.8% 


27.9% 


4 


18 


13.3 


26.6% 


21.6 


43.2% 


16.6% 


5 


34 


8.8 


17.6% 


24.3 


48.6% 


31% 


8 


20 


13.4 


26.8% 


24.3 


48.6% 


21.8% 


7 


25 


13.6 


27.2% 


26.6 


53.2% 


26% 


8 


17 


10.2 


20.4% 


19.3 


38.6% 


18.2% 


9* 


19 


16.7 


33.7% 


25.4 


50.8% 


17.1% 


Total 


163 


115.5 




210.8 






Average 


163 


12.9 


25.8% 


23.4 


46.8% 


21.6% 



Follow-Up Workshop 
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Chapter 7 



Results of the Fo llow-Up Instructors Survey 

During the field test of the provider's workshops, eight workshops were conducted by 12 of the 18 
individuals who had participated in the June 5 and 6, 1989 instructors workshop. To assess their 
perceptions, a survey was sent asking for information about their experience and their assessment 
of the training and the materials. (See Appendix G.) Results of that survey are presented below. 

Question One: How would you improve the Instructor's Workshop? 

The Instructor's Workshop was held during a two day period. The agenda followed the same 
sequence as the Participant's Coursebook with the addition of enrichment activities. Suggestions 
from respondents regarding improving the workshop included: 

• Provide opportunities to practice teaching the material. 

• Expand Workshop to 3 days. 

• Allow more time for discussion and questions. 

Question Two: How did the instructors perceive their own knowledge of the information 

they were presenting during the workshop? 

All of the respondents (N=5) felt very comfortable regarding their knowledge of developmental 
disabilities. All but one was comfortable with their knowledge of fire safety and questioned their 
preparation regarding licensing certification standards. Respondents felt most comfortable 
teaching the module on fire hazards, and least comfortable teaching about fire behavior and fire 
emergency planning. 

Question Three: Which modules did instructors feel prepared to teach? 

Respondents (N=5) felt most prepared to teach the module on fire hazards and least prepared to 
teach the modules on fire behavior and fire emergency planning. Only one of the respondents felt 
unprepared to teach any of the modules. 



Table 7-1 

Modules in Order of Teaching Comfort of Instructors (N=5) 



Module 


Completely 


Somewhat 


Not Prepared 


Fire Hazards 


80% 


10% 


0% 


Introduction 


60% 


40% 


0% 


Life Safety Standards 


60% 


40% 


0% 


Human Factors 


60% 


40% 


0% 


Fire Safety Devices 


60% 


40% 


0% 


Resident and Staff 
Training 


40% 


60% 


0% 


Fire Behavior 


40% 


40% 


20% 


Fire Emergency 
Planning 


20% 


80% 


0% 



Question Four: How would you improve the Instructor's Manual? 

The Instructor's Manual had been 3-hole punched and placed in a 3 ring binder. The materials 
consisted of approximately 20 "pink pages," materials that explained the purpose of each module, 
with suggestions for introducing each one, answer keys for some of the activities in the 
coursebook, and additional technical information. The Manual also included annotated pages of 
all of the activity pages of the Coursebook. The Coursebook materials were reduced. At the top of 
each page were the activity objective and instructions for presenting the activity. In the right 
margin were Key Points and Interesting Facts. 

Of the respondents (N=5), all strongly approved (80%) or approved (20%) of the format used to 
publish the Instructor's Manual including the two color format, the annotations, used to publish the 
Manual. Overall, they found the Manual easy to follow and teach from. Although all of the 
respondents approved of the print size, their enthusiasm was less with 20% strongly approving, and 
80% approving. Respondents had no suggestions for improving or changing the format or the 
content of the Instructor's Manual. 

Question Five: Which training aids were used and how did participants respond? 

All of the respondents (N=5) used all five of the training aids. Table 7-2 below presents the 
instructors' evaluation of training aids from most to least preferred. Of the training aids, the 
Human Behavior in Fires, audiocassette was least preferred. 



Table 7-2 

Instructor's Evaluation of Training Aids After Conducting Workshops (N=5) 



Training Aid 


Used 


Very 
Good 


Good 


Fair 


Poor 


NR 


1. Flashover: Point of No Return 


100% 


40% 


40% 


0% 


0% 


20% 


2. Participant's Coursebook 


100% 


40% 


40% 


0% 


0% 


20% 


3. Need for Fire Safety (video) 


100% 


40% 


40% 


0% 


0% 


20% 


4. Job Aids (Checklists) 


100% 


20% 


60% 


0% 


0% 


20% 


5. Human Behavior in Fires (audiocassette) 


100% 


0% 


20% 


40% 


0% 


40% 



Question Six: What additional training aids did the instructors want? 



Instructors were asked if they needed any additional training aids. They were given choices of 
overhead transparencies of job aids, additional audiocassettes, and additional videos. Space was 
provided for them to add their own suggestions. Some of the respondents thought that additional 
aids were not needed and that they would not have time to use any additional materials. Other 
respondents said they would like overhead transparencies (N=l), and additional videos (N=2). 
Other suggestions were fire safety posters, videos demonstrating escape routes and stop, drop, and 
roll. 

Question Seven: Did the instructors use a flipchart or chalkboard as a training tool? 

All of the respondents reported using either a flipchart, a chalkboard, or both as training tools. Of 
the respondents (N=5), 40% used a flipchart, 20% used a chalkboard, 40% used both. 

Question Eight: Did the instructors use the games and how did participants respond to them? 

Instructions and materials were provided for instructors to conduct two games during the 
workshop: Charades and Quickdraw. In both cases fire safety devices and terms were used as the 
content for the games. 




Of the respondents (N=5), 40% used both Quickdraw and Charades, 20% used only Quickdraw, and 
40% used neither. Instructors reported that participants were hesitant at first but eventually 
enjoyed playing the games. Instructors cited lack of time as the reason for not using both or either 
game. When asked if they would use the game in the future, 80% said yes if time permitted. 

Question Nine: How could the Participant's Coursebook be improved? 

All of the respondents (N=5) preferred a permanent binding rather than the 3 hole punch, shrink 
wrapped format used in the field test They also suggested numbering pages consecutively 
throughout the book. One suggested that the fire safety floor plans should be expanded, and one 
person suggested that the pictures in Module 2, at the bottom of page 5 on safe egresses be explained. 

Question Ten: How did the instructors feel about their teaching experience? 

All of the respondents (N=5) felt that they would find teaching the workshop easier the next time. 
Only 40% said they were willing to conduct workshops other than for Project Home providers; the 
remaining 60% were uncertain. 

Question Eleven: How did instructors perceive the interest and learning of the participants? 

All of the respondents (N=5) felt sure that the participants were interested in fire safety and 
learned a great deal during the workshop. 

Question Twelve: How did the instructors perceive their own knowledge of the information 
they were presenting during the workshop? 

All of the respondents (N=5) felt very comfortable regarding their knowledge of developmental 
disabilities. All but one was comfortable with their knowledge of fire safety and questioned their 
preparation regarding licensing certification standards. Respondents felt most comfortable 
teaching the module on fire hazards, and least comfortable teaching about fire behavior and fire 
emergency planning. 

Question Thirteen: How did instructors judge the facilities where workshops took place? 

Respondents were divided on this issue; two of them were dissatisfied and two were satisfied. 
Problems cited were: noisy air conditioning, room too large, room too small, no tables for 
participants to write on. 

Question Fourteen: Did participants ask any questions that instructors could not answer? 

Only one question: "Why are fire extinguishers required if BWA recommends not using them?" 
was reported by respondents (N=5). 

Question Fifteen: What ideas did instructors have about improving the workshop for 
providers? 

Respondents (N=5) consistently felt there was too much to be covered during the time allotted and 
suggested holding the workshop over a two day period. No respondent suggested reducing the scope 
of the workshop or eliminating information from the coursrbook. Participants in the field test 
were given only short breaks and no lunch hour. Respond .fe -uggested lengthening the 
workshop time and including longer breaks and an hour fo. i .,ch. Respondents also suggested 
reducing the time required to administer the post test. 




Question Sixteen: Have any providers asked to use training aids? 

In order to implement Module 7, Resident and Staff Training, workshop participants need to 
borrow audiovisual training aids from the certified instructors. Respondents reported that none of 
the providers to date (approximately 6 to 8 weeks after the workshop) had asked to borrow 
materials. 

Question Seventeen: Were the instructors planning to subscribe to the Continuing Education 
Program ? 

Of the respondents (N=5), only two planned to subscribe to the CEP. The remaining instructors 
cited lack of funds, and uncertainty about the opportunities to use the materials. 




Chapter 8 

Summary of Findings , Problems, and Recommendations 
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Purpose of the Study 

The purposes of this study were: 

1. To develop a program for training instructors who could effectively present the Fire 
Safety Workshop for Board and Care Operators and Staff and to pilot test that program. 
The program included a two day workshop. The materials included an agenda and 
program for the workshop, a set of evaluation materials, an instructor's manual and 
other items previously developed for the Fire Safety Workshop. 

2. To field test the one day Fire Safety Workshop for Board and Care providers using 
the instructors who had completed the pilot test of the Instructor's Workshop. 

Instructor's Workshop 
Data sources 

Preregistration forms, pre- and post tests, course evaluation, materials evaluation, follow-up 
instructors survey, transcript of video recording of workshop, observations of staff at the 
workshops, 

Summary of Findings 

1. Did the participants improve from pre- to post test individually and as a group? 

Participants (N=16) in the Instructor's Workshop improved their mean score as a group from 
approximately 4<K& to 77%. The gain of 37 percentage points was statistically significant at greater 
than the .05 level of confidence. 

2. Did a majority of the participants improve on a majority of the items? 

Participants' performance on the 21 test items (each drawn from the same content domain) 
improved from pre- to post on all but three items. 

Item 8 on the pretest asked participants to name resident activities that might impact on fire safety 
The post test item asked them to list characteristics of staff that might impact on fire safety 
Participants generally could answer the question about residents, before the workshop; but could 
not identify staff characteristics at the conclusion of the workshop. 

Item 9 on the pretest asked participants to name two resident characteristics that would impact on 
fire safety. Most participants were able to identify at least one correct answer. On the post test the 
participants were asked to identify two characteristics of residents with mental disabilities that 
might impact on fire safety. The majority of participants were unable to respond with even one 
correct answer. 

Finally, Item 12 of the pretest asked participants how to reduce the risk of electrical fires All but 
one person was able to respond with a correct answer. Item 12 on the post test asked how to reduce 
the risk of appliance fires. Only one participant was able to answer Item 12 correctly. 
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Responds to Items 1 and 3, testing knowledge of Module One, Fire Behavior, the topic that 
participants felt least comfortable teaching, indicates that participants actually improved their 
knowledge on content draw from that domain. 

Responses to Items 5, 17, 18, 19, and 20, testing knowledge of Module Six, Fire Emergency 
Planning, also indicates that participants were prepared to teach that content even though they felt 
less comfortable teaching this module than others. 

3. Did the participants improve from pre- to post test on individual items? 

A majority of the participants (15 of 16) showed improvement on a majority of the items (11 or 
more). On the average participants improved or stayed the same on 19 of the 21 items. 

4. Did the participants have positive feelings about the workshop and their learning 
experience? 

A large majority of participants (86%) agreed or agreed strongly with each statement about the 
workshop. Statements which received the least positive responses related to participants having a 
clear idea of how to present the workshop and an understanding of licensing and certification 
standards. 

5. Did the participants feel the training aids were useful in helping present the Workshop? 

Overall, 69% of the participants rated the training aids either excellent or good. Generally, they 
strongly approved of the videotapes Flashover: Countdown to Disaster and Flashover: Point of No 
Return. They gave the video "Need for Fire Safety" a strong approval with 76% rating it excellent 
or very good. The audiocassette "Human Behavior in Fire" was rated excellent or good by 59% of 
the group and average by 29%. The videos "Silent Response" and "Everybody Needs a Buddy" 
were acceptable to most of the participants. The film "Know Your Fire Extinguisher" received the 
least positive ratings. Participants cited its out-of-date information as the reason for their poor 
rating. 

6. How did the workshop training vary from the plan in terms of time spent on each topic? 

Because the workshop started late on the first day and the time used to present Module One, Fire 
Behavior, exceeded the allotted time, all of the other modules taught during Day One were given 
less time that had been allotted. During Day Two, the 33 additional minutes spent on Module 4, 
Fire Hazards, reduced the time planned for the other modules. 

7. How did the scope of the content vary from the planned presentation? 

The content did not deviate markedly from the information in the Coursebook and Instructor's 
Manual. Issues brought up that were not in the planned scope were: arson, legal liability, 
woodstoves, techniques firefighters use to fight fires, summarizes of Chapters 17, 20, and 21 of the 
NFPA Life Safety Codes, frying foods as a fire hazards, insulation of fireplace chimneys, 
deadbolts, location of smoke detectors (wall, ceiling), advantages of hard-wired smoke detectors 
versus battery-operated detectors, gas leaks. Nearly all of these diversions were in response to 
questions from the participants. 
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Follow-Up Instructors Survey 



Data Sources 

Data were gathered from survey forms sent to each instructor who completed a workshop. 
Summary of Findings 

1. Instructors asked for opportunities during the Workshop to practice teaching the materials 
and more time for discussion and questions. One respondent suggested expanding the workshop 
to 3 days. 

2. Overall instructors felt comfortable regarding their knowledge of the information they 
had to teach and with the skills they were expected to have. They felt most comfortable with the 
module on fire hazards and the least comfortable teaching fire emergency planning. 

3. Respondents generally strongly approved of the Instructor's Manual format and 
information. They found it easy to follow and teach from. 

4. Respondents used all of the training aids. They strongly approved of the videos 
"Flashover: Point of No Return," and "The Need for Fire Safety" as teaching aids. They also 
strongly approved of the Coursebook. They approved less strongly of the Job Aids and generally 
were unenthusiastic about the audiocassette "Human Behavior in Fires." 

5. Additional training aids suggested by instructors included additional videos about fire 
drills and stop, drop, and roll, and fire safety posters. One respondent wanted overhead 
transparencies of job aids. 

6. All of the instructors responding to the survey used either a flip chart, a chalkboard or both 
as training tools. 

7. The games developed for the workshop were used by 60% of the respondents. Those who did 
not use them cited time as the reason. 

8. All of the respondents preferred a permanent binding, rather than the 3 hole punch loose 
leaf format used in the field test. They suggested numbering pages consecutively rather than by 
module. 

9. Overall, instructors believed that they would find teaching the workshop easier the next 
time. Of those responding, 40% were willing to teach the workshop to providers other than for 
Project Home. 

10. All of the respondents believed that participants were interested in fire safety and learned 
during the workshop. 

11. Overall, the respondents were comfortable regarding their knowledge of topics covered by 
the workshop. They felt most comfortable teaching about fire hazards and least comfortable 
teaching about fire behavior and fire emergency planning. 

Provider's Workshop 
Data sources 

Preregistration forms, pre- and post tests, course evaluation, materials evaluation, 
observations of staff at the workshops. 



Summary of Findings 



1. Did the participants, as a group, improve from pre- to post test? 

Participants' mean score improved an average of 12 points or 24% from pre- to post task This 
difference was statistically significant at a greater than an .05 level of confidence 

2. Did the participants improve their scores from pre- to post test on each item? 

As a group, the participants improved at least slightly on all but Item 5 which tested the participants 
knowledge about alternate escape routes. As a group, they approved an average of .57 points per 
item. 

3. Did the participants improve from pre- to post test on individual items? 

Results showed that 46% of the participants had improved on a majority of items (11 or more). As a 
group, the participants improved on approximately 10 items, stayed the same on 8.5 items and 
received lower post test scores on 2.5 items. Those who took Form B as a pretest and Form A as a 
post test performed slightly better, improving on 1.4 more items than the AB Group. 

2. Did the participants have positive feelings about the workshop and their learning 
experience? 

Results suggest a strong approval by participants and a believe that they had gained knowledge of 
fire safety. Approximately 90% of the participants either agreed or strongly agreed with each 
statement about the training. Items 2 and 3 which dealt with identifying fire safety problems and 
making their facilities safer received the most positive ratings. Item 14 which dealt with the 
Continuing Education Program received the least positive ratings, with 59% of the participants 
agreeing or strongly agreeing that the program would be useful to them. 

6. How did the workshops vary from the plan in terms of the time spent on each topic? 

Information gathered by observers at each workshop showed that the largest block of time was spent 
on Module 1, Fire Behavior. Participants took longer amounts of time for breaks than had been 
planned. Instructors also spent more time on Module 2, Life Safety Standards, than had been 
planned. All of the other modules were given less time than had been planned with the greatest 
difference for Module 6, Fire Emergency Planning. 

7. Did the participants show improvement on each module? 

Participants as a group showed positive improvement on all modules. The module with the most 
improvement overall was Module 5, Fire Safety Devices. Items on which they showed the greatest 
improvement (in order) those that measured their knowledge about : vertical openings, 
emergency tasks for residents, smoke detectors, fire drills, and fire behavior. The topic on which 
they performed the worse was included in Module 6 and measured knowledge about alternate 
escape routes. 




Provider's Follow-Up Workshop 



Data Sources 

Data were collected from preregistration, pre- and post tests, course evaluation, provider's survey 
forms, and observations from a BWA staff member. 

Summary of Findings 

1. Did the participants improve from pre- to post test? 

Participants improved an average of 10.6% from pre- to post test. The difference was statistically 
significant beyond the .05 level of confidence. 



2. Did the participants have positive feelings about the workshop and their learning 
experience? 

Over 87% of the participants either agreed or strongly agreed with each positive statement about the 
workshop and their learning experience. They strongly approved of the instructors and the 
Coursebook. 

3. How did the participants' performance in the Folio w-Up Workshop compare with the 
performance of the participants in the Field Test Workshops? 

Although the average post test score for participants in the follow-up group was higher than for the 
field test group, the gain score was less due to the higher pretest scores of the group. Participants in 
the follow-up group viewed the video "The Need for Fire Safety* before taking the pretest which 
may have affected that outcome. 
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Recommendations for the Instructor's Workshop 



Introduction 

The pilot test of the Instructor's Workshop took place over a two day period. The format consisted of 
a presentation of the Provider's Workshop by four different instructors, each of whom were experts 
in the specific content area of the modules they presented. Interspersed with the fire safety modules 
were presentations about training techniques and other information instructor's needed to present 
the workshop such as administering the evaluation instruments, using training aids. 
Participants in this workshop viewed all of the training aids designed for the Provider's 
Workshop plus two 16mm films and two additional videotapes. 



Procedures 

Information from all sources was collated and analyzed for overall approval or disapproval of the 
workshop. Data included information collected before and after the workshop and after 
participants had presented one or more sessions of the workshop to providers. 



Problems Identified 

1. The average post test score of the participants in the instructor's workshop was 77%. This 

score, if it is a reliable measure of their learning, needs to be improved. 

2. Participants, after having conducted a workshop, suggested adding opportunities to 

practice presenting the materials before actually conducting a workshop with 
providers. 

3. Participants wanted workshop instructors to model more closely the manner in which the 

workshop should be taught to providers. 

Recommendations 

1. Revise the agenda (format) of the Instructor's Workshop to include: 

Day 1: Present the Workshop in the same time frame and with the same training aids 
in the same manner as the instructors are expected to present it to providers. 

Evening: Conduct an optional evening session to provide participants an opportunity to 
practice teaching parts of the Workshop with assistance from Certified 
Instructors. Videotape instructors. 

Day 2: Conduct enrichment activities regarding! fire safety, board and care 

operation s,needs and characteristics of people with development disabilities, 
training techniques, working with adults, and administrative tasks associated 
with presenting the workshop (e.g., administering tests). 

2. Videotape a model session of the Provider's Workshop (possibly videotape Day 1 of the 
Instructor's Workshop) and make copies of this tape available to instructors. 

3. Send participants a copy of the Provider's Workshop Coursebook in advance of the 
workshop along with other materials that they need to become familiar with. 

4. Go over results of the pretest and be sure to answer any questions participants have about the 
content of the course prior to the administration of the post test. , 
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6. 
7. 
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Establish a criteria for passing the post test. Participants who are unable to demonstrate \ ' ' / - 
mastery of the content should not present the workshop or be certified as instructors. rrr^' -^ T-"^ 



Redesign the post test to make it a more reliable instrument 



Make the workshop more interactive, encourage participation by asking them questions, 
waiting for answers, and asking them to ask questions. Be sure participants complete each 
exercise in the Participant's Workshop as they will be expecting provider's to do. 
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Include instruction on operating the videotape recorder and audiocassette. 
IndiLde^ilLdemonstmtiQn_Q^the g^Ps tfinfr »r* ^ g i n ^ 0 r,t^wo^h5>p r 

Stress use of Coursebook activities and training aids by providers with their staff and 
residents. 
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Chapter 10 

Recommendations for the Instructor's Manual 



Introduction 

The pilot test version of the Instructor's Manual was 3 hole punched and packaged in a 3 ring 
binder. Each module was separated with lavender, cover stock, oversized divider pages. The 
Instructor's Manual was 143 pages, printed on one side only. 

The manual consisted of two types of materials: 

1. A reduced version of the activities pages from the Participant's Coursebook printed in 
black ink with instructions for presenting the information in red ink at the top of the 
page. To the right of the reduced coursebook, also in red ink, were key points and 
interesting facts to further guide the instructors in presenting the materials. 

2. In addition to the activity pages, the instructor's manual also had "pink" pages, 
instructions for introducing the modules, response sheets to activities in the 
Coursebook, and additional information on various subjects. 

Data Sources 

Information used to recommend changes to the Instructor's Manual were gathered from three 
main sources including: Written notes from BWA observers during the workshops, the transcript 
of the videotape of the June 26, 1989 Focus Group meeting during which time members discussed 
problems and solutions , and the follow-up survey of instructors. 

Procedures 

Based on the analysis of all data, project staff prepared recommendations for specific changes to 
the Instructor's Manual. A draft of the revised Manual and a list of specific changes made to each 
page were submitted to members of the Focus Group for a full discussion at the Focus Group 
meeting on September 8, 1989. 

Recommendations 

1. Change the Instructor's Manual to match changes to the Participant's Coursebook. 

2. Continue publishing the annotated coursebook edition with two-colors. 

3. Where they do not presently exist, add specific questions that the instructors are expected to 
ask participants. 

4. Use a different type style for the Instructor's notes to make it stand out more. ' ' 

5. Improve instructions on using the Coursebook activities and materials with staff and 
residents. 
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Chapter 11 

Recommen dations for the Provider's Workshop 



Introduction 

The agenda for the Provider's Workshop was developed by BWA and Project Home. Participants 
were recruited by Project Home. Attendance at the workshop was required for renewal of the 
provider's certification. The agenda included three short breaks. Participants were expected to 
break briefly for a snack in the middle of the day and to eat lunch while the workshop continued. 
Workshop sites were selected by Project Home. No guidelines for selected a site or limiting the 
class size were given by BWA prior to the workshops. 

Data Sources 

The primary sources of data for the evaluation of the Provider's Workshop were written notes of 
observers at the eight field test sites, comments of the Focus Group members during the June 26, 
1989 meeting, and the Instructor's Follow-up survey. 

Recommendations 

1. Reduce the amount of material to be covered during the Workshop, especially the 
information in Module One, Fire Behavior. 



2. 



Reduce the amount of time required to administer the pre- and post tests;^~> - 
Extend the agenda to allow for a full hour for lunch. — 5 S T/.i^^V^ 



3. 
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4. 



Make the workshops more interactive, encourage provider participation. 



5. 



Select a site where tables are available for participants to work on during the training. 



6. 




7. 



8. 
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Recommendations for the Provider's Coursebook 



Introduction 

The field test version of the Participant's Coursebook consisted of 120 pages, printed front and 
back, 3 hole punched, and individually shrink-wrapped. Included in those pages were the front 
and back covers (4 pages), and seven divider pages with a table of contents for each section (14 
pages). See Appendix B for the complete Table of Contents. The Coursebook included 42 black and 
white photographs and 19 graphics, including line art illustrations and charts. The cost of 
printing 400 copies for the pilot test of the instructor's workshop and field test of the provider's 
workshop was $3,400. The cost to print each book was $8.50 not including editing, photography, and 
writing. 

Data Sources 

Data from all sources were collated for each page of the Participant's Coursebook including the 
pre- and post test results and course evaluation results for items measuring information on each 
page, and observer (N=8) recommendations and comments. Also included were written notes by 
observers during the eight field test workshops and follow-up workshop, information gathered 
from instructors on the Instructor's Follow-Up Survey, information from the transcripts of the 
videotape of the Instructor's Workshop and the Follow-up Workshop. Information was also sought 
by an independent review by an expert in fire safety, board and care fires, and Chapter 21 of the 
NFPA Life Safety Code. 

Procedures 

Based on the analysis of all data, project staff prepared recommendations for specific changes to 
the Participant's Coursebook. In addition to specific changes, it was the consensus of the Focus 
Group members and project staff, that the number of pages of the Coursebook should be reduced 
from 120 to 96 in order to reduce the cost of printing. The amount of information to be presented at 
the Workshop also needed to be reduced either by reducing the information in the Coursebook or by 
changing the workshop presentation. 

A draft of the revised Coursebook and a list of specific changes made to each page were submitted to 
members of the Focus Group for a full discussion at the Focus Group meeting on September 8, 1989. 
Final changes to the materials are listed by page in Appendix C. A summary of the changes are 
presented below in Table 12-1. In addition to specific changes for each module, overall changes to 
the format and presentation are presented in Table 12-2. 
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Table 12-1 

Summary of Proposed Changes to the Provider's Coursebook by Module 



Module 


Changes 


Introduction 


Change to Module 1; Add a page on liability issues, reduce Fire Hazard News 
(pages 1 and 2) to one page; Add additional resources. 


Fire Behavior 


Reduce number of pages and limit scope to an understanding of the four 
components of fire: fuel, heat, oxygen, and chemical process; plus how fires 
start, smoke and flashover. Reorder pages so the last activity in the module 
will be viewing and discussing the video Flashover: Point of No Return 


Life Safety Standards 


Organize materials so that all standards that apply to small homes are on one 
page; all standards that apply to large homes are together. Develop a matrix 
with the 3 types of groups on one axis (Prompt, Slow, Impractical) and 6 areas of 
standards as the other matrix. Add information on the danger of using 
deadbolts. Remove the Residence Certification Standards Checklist. Move to 
page before Human Behavior in Fires. Include one checklist for large and one 
for small facility. Add a page giving suggestions for setting up a designated 
smoking area after the information on interior finishes and interior 
furnishings. 


Human Factors in 
rire oatety 


Add a page on AIDS patients; End the module with the audiocassette a Human 
Factors in Fire Safety." Change the activity page so that participants check off 
each point as they hear it on the tape. Move Community Resources. Start 
section on disabilities with summary activity (page 13 in field test edition). 
Add a photograph or line art to staff and resident activities pages 


rire nazarcis 


Add information on heating pads and electric blankets; Add information on 
wood stoves, as well as how to set up a designated smoking area 


Fire Safety Devices 


Change the format of page 1 to devices they should have, devices recommended 
for large homes, and special devices. Participants will check off the devices 
they have before starting the Fire Safety Devices Module. 


Fire Emergency 
Planning 


Remove floor plan pages; Add a page on planning an alternate route. 


Resident and Staff 
Planning 


Reduce from 5 to 4 pages 


CEP Information 


Include CEP information and order form on last two pages of Coursebook 



Table 12-2 

Summary of Overall Changes to the Provider's Coursebook Format 



1. Print without section dividers to save on cost of printing. 

2. Select a permanent binding rather than loose leaf to provide for easier handling in Workshop 
If possible, continue to print on 3 hole punch paper. Use perforation for the job aid pages. 

3. Number the pages consecutively throughout the book starting with Module One The Need for 
Fire Safety. ' 

4. Add line art and photographs wherever needed to make pages more attractive. 

5. Review all information pertaining to the NFPA standards. Change sections titled "What are 
the NFPA standards...." to "A Good Idea." Include recommendations for maximum safety 
without regards to whether they are specifically included in NFPA code. 
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Chapter 13. 

Recommendations for the Evaluation Instruments 



Introduction 

The two primary instruments used to assess participant learning and affect towards the materials 
were the pre- post test and the course evaluation. The pre- post test were designed to be two 
equivalent forms consisting of 21 items each of which measured a domain of knowledge. 

Procedures 

Data from all sources were collated to identify problems related to the pre- post test and to 
determine the appropriate methods of solving those problems. Based on the analysis of all data, 
project staff prepared recommendations for specific changes to the post test. A draft of the revised 
test containing 63 items were submitted to members of the Focus Group review. 

Summary of Problems Identified 

1. Of the 249 number of individuals who attended the field test workshops, 200 attempted to 
complete a pretest, and 98 completed the entire test. A large percentage of the population, 
49%, either did not attempt the pretest or did not complete it during the time allotted. At most 
workshops, observers noted that there were people who seemed unable to read the test items. 
Participants were observed telling the instructors that they had left their glasses home. In 
some cases, participants arrived late and did not have the same amount of time as others to 
complete the test. 

2. Of the 249 individuals who attended the workshops, 175 attempted to complete a post test, and 
157 completed the entire test. Of those who attended 74 did not complete a post test even 
though they were told that they would not receive a certificate if they did not complete the 
test. In some cases participants left without either attempting or completing the test citing a 
lack of time or commitments with clients. 

3. It is difficult to develop a reliable testing instrument with open-ended questions even when 
an answer key has been prepared. Participant responses varied from workshop to 
workshop depending on the interpretation and emphasis of the instructors. 

4. In the follow-up workshop, participants were told that they could use the Coursebook during 
the post test 

5. The tests were too long and took too much time from the instructional period of the 
workshop. 

6. The gain from pre- to post test was probably affected by the fatigue of the participants at the 
end of a six hour workshop with only short breaks. 

7. The Workshop Evaluation did not have an item which matched each module. 

Changes to the Post Test were needed to improve readability, increase the reliability, and to reduce 
the amount of time required to complete the test. 
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Recommendations 

1. Develop a multiple choice test with two equivalent forms which can be used as a pre- or a 
post test 

2. Administer the test by reading each item out loud and having participants respond by 
circling the correct response. 

3. To add construct validity, include a section that measures the participants' values related 
to fire safety and to their plans for future action related to fire safety. 

4. To add predictive validity, develop a follow-up testing procedure to determine what 
changes the participants have made in their fire safety routines following the workshop. 
Compare results to information obtained from the Provider's Survey. 

Proposed Changes to the Workshop Evaluation for Providers 

Changes to the workshop Evaluation for Providers were needed to improve readability, and to 
increase the readibility. 

Recommendations 

1. Develop a new set of evaluation questions, so that items measure the participants feelings 
towards the instructors, the atmosphere, and the course. 

2. Use a Likert-type scale, with the middle response being "No Opinion" instead of 
"Uncertain." 

3. Administer the evaluation by reading each item out loud and having participants respond 
by circling their response. 

Proposed Changes to the Workshop Evaluation for Instructors 

Changes to the workshop Evaluation for Providers were needed to improve readability, and to 
increase the readibility. 

Recommendations 

1. Develop a new set of evaluation questions, so that items measure the participants feelings 
towards the instructors, the atmosphere, and each module of the course. 

2. Use a Likert-type scale, with the middle response being "No Opinion" instead of 
"Uncertain." 

3. Add additional items about which materials the participants will feel confident teaching 
in the workshops they will conduct. 
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Chapter 14 
Implications for Phase 111 



Introduction 

Phase I and Phase II of the project to develop a Fire Safety Certification System for Board and Care 
operators has been funded by Small Business Innovation Research contracts from the Office of 
Human Development's Administration on Developmental Disabilities. Phase III activities must 
be self-sustaining in terms of funding. 

Phase III Goals 

The goals during Phase III are: 

1. Revise the Workshops and the materials to include the recommended changes and publish 
a supply of the materials adequate to meet the needs for approximately one year. 

2. To the greatest extent possible, encourage licensing and certifying agencies, associations, 
large provider organizations, members of fire departments who conduct training, and 
others to enroll in the workshops to train instructors and to then conduct training for 
providers on a regular basis. 

3. Receive additional grants from organizations that fund fire safety activities such as the 
Tobacco Institute, manufacturers of fire safety equipment. 

4. Recruit members for the Continuing Education Program in sufficient numbers to make 
that activity seif-sufficient-500 members during the first publication cycle. 

5. Conduct follow-up research using the population of providers who have completed the 
workshop to determine how the training affected their attitude and behavior with respect to 
fire safety. Use a control group of providers who have not experienced the workshop. 

6. Analyze workshop results to determine characteristics of instructors that seem to be 
associated with the higher results. 

7. Seek peer review by: 

a . Developing reports and articles based on the Phase I and Phase II experiences and 
data for publication in professional journals. 

b. Develop papers for presentation at conferences and meetings of professional 
associations. 
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